FeedCreator and utf-8 specialchars

Site Admin Joined: Aug 22, 2007 Posts: 1772

Hey guys!

I have a problem with the FeedCreator and specialchars of the news title, if i using utf-8 as charset and in my database.

utf-8 issue: backend.php
open: includes/feedcreator/feedcreator.class.php

search near line 984:

Code:

$feed.= "            <title>".FeedCreator::iTrunc(htmlspecialchars(strip_tags($this->items[$i]->title)),100)."</title>\n";

I can't add ENT_QUOTES after 100, because it doesn't work. So tried the following line and it doesn't work, too:

Code:

$feed.= "            <title>".FeedCreator::iTrunc(htmlspecialchars(htmlspecialchars_decode(strip_tags($this->items[$i]->title), ENT_QUOTES), ENT_QUOTES, _CHARSET),100)."</title>\n";

After this one i tried the following line and it works, but only with utf-8. It makes me confused, why i must use here utf8_decode, if i have an utf-8 charset and an utf-8 database and xml feeds have normaly an utf-8 charset, too. where is here the problem:

Code:

$feed.= "            <title>".FeedCreator::iTrunc(htmlspecialchars(htmlspecialchars_decode(utf8_decode($this->items[$i]->title), ENT_QUOTES), ENT_QUOTES, _CHARSET),100)."</title>\n";

Posted: Sat Jun 30, 2012 1:44 pm

This is my last problem for my news mod and i have no idea to fix it. Here i need a answer from the module author. Giv it no other way to fix the specialchar problem in the feeds. I think the problem is inside all other feeds, too.

Posted: Sun Jul 01, 2012 7:55 am

neralex, can you help us understand better what the issue is that you are running into? You only mention the cause, but not really the issue.

Thanks.

Posted: Sun Jul 01, 2012 10:25 am

Yes of course!

I have stored the news title with the specialchars "äüö&ß" in my utf-8 database with this line:

Code:

$subject = $db->sql_escape_string(htmlspecialchars_decode(check_html($subject, 'nohtml'),ENT_QUOTES));

In the databe is all fine and it shows "äüö&ß" without html code.

If i look now in the xml feed of the backend.php, then i get this result:

Code:

Ã¤Ã¼Ã¶&ÃŸ

I have tested with the ISO charset and a utf-8 database and there are no problems. Only the combination with utf-8 as charset and an utf-8 database result this issue. In the Feeds module with the html-view of the feed is all fine. Here works the filtering.

With the following fitlering is all fine but it doesn't works with a ISO charset:

Code:

$feed.= "            <title>".FeedCreator::iTrunc(htmlspecialchars(htmlspecialchars_decode(utf8_decode($this->items[$i]->title), ENT_QUOTES), ENT_QUOTES, _CHARSET),100)."</title>\n";

The xml file have a utf-8 charset and normaly i don't need the utf8_decode to show the specialchars, if i use an utf-8 charset and a utf-8 database. I get this problem only in backend.php. Is it possible here works an ISO filtering on a other place?

Quote:

utf8_decode — Converts a string with ISO-8859-1 characters encoded with UTF-8 to single-byte ISO-8859-1

Posted: Tue Jul 03, 2012 5:03 am

Couple of questions to help me better understand this..
Assuming your website is using utf-8 and your database is also utf-8 (not utf-8_bin) what field type is the data being stored in?
I ask because I have had instances when converting existing tables from latin1 to utf-8 the feild type has changed to binary Blob.

Posted: Tue Jul 03, 2012 9:44 am

the RN have charset utf-8 and the whole nuke database was created with utf8_general_ci. The database was not converted, its a fresh install with utf8_general_ci in all fields. I use the the utf-8 versions of db.php and mysqli.php from Palbin.

Palbin wrote:

Try these two files:
db.php - [ Only registered users can see links on this board! Get registered or login! ]
mysqli.php - [ Only registered users can see links on this board! Get registered or login! ]