XML parsing error in backend.php

Susann · Posted: Wed Jun 06, 2007 5:40 pm

I just noticed that I can´t use words with ü in my site describtion anymore because the result is a XML parsing error.
I´m using this word there since years Rolling Eyes

and I believe in my previous fixed version I didn´t get an error for this. Hope I can find the file somewhere again. I simple renamed the word and used "ue" instead of "ü" but thats not a permanent solution.

Gremmie · Posted: Wed Jun 06, 2007 6:36 pm

Does this work instead:
ü

Question

Susann · Posted: Wed Jun 06, 2007 6:59 pm

Thanks Gremmie I changed that under general settings and that works for the backend.php but thats not the correct way to solve this issue because this word would not show correctly in Search Engines.

Gremmie · Posted: Wed Jun 06, 2007 9:05 pm

I'm certainly not an expert on internationalization, but isn't there a way to change your charset by inserting something in the header or even the DOCTYPE?

Susann · Posted: Thu Jun 07, 2007 2:47 am

It`s solved. I simple changed the word in nuke_config back to ü and the backend.php passed validation.

montego · Posted: Thu Jun 07, 2007 5:56 am

Well, that doesn't make sense to me that just putting it back the way it was fixed it. It must not be exactly the same. I am confused... Anyways, glad its working now.

To be honest, I am anxiously awaiting kguske's completed work on nukeFEED as I am pretty sure that would be the better way to go overall and we would want to eventually get rid of the old RSS methods.

kguske · Site Admin Joined: Jun 04, 2004 Posts: 6044

I'll use ü in a test to see how well it works and if it validates. Development of nukeFEED is coming along - I hope to have a working version on the site soon - right now, it's on a test system only. I copied nukeSEO to a test system just so I could test situations like this - different words that wouldn't necessarily occur on the existing site.

montego · Posted: Thu Jun 07, 2007 7:03 am

kguske, definitely interested to see what you come up with. There are going to be issues with chopped off entities too, where you may end up with just a "&" or a partial "&am" or something like that. Raven got around some other issues by converting many htmlentities into their encoded decimal equivalents (see the two backend.php scripts in latest RN - might want to check the SVN update logs to see the progression).

I did some more work on it adding some CDATA.

But, you probably already knew this stuff and are way beyond that... Wink

Susann · Posted: Thu Jun 07, 2007 5:21 pm

Montego to explain this issue better its just so that I use a word with ü lets say its "Bücher" for books and this word is in the administration written as Bücher. So far so good.
The backend.php validates because in the db table nuke_config there slogan its written the same way just "Bücher" but as soon as I change something in the Nuke administration it updates the db and the word has automatically changed there to "Bücher" and the backend.php doesn´t validate. Undefined enity is the error.
So is there a way to fix this permanently ?

Gremmie · Posted: Thu Jun 07, 2007 6:54 pm

I'm guessing that htmlentities is turning your ü into ü. I need to do some more reading about this but I will look into it.

montego · Posted: Thu Jun 07, 2007 8:24 pm

Looks to me like we need to make more complete use of the entity_to_decimal_value() function with the backend scripts. I had no idea that we would get html entities in the slogan file, but it sounds like it could possible happen in sitename and others too.

Thanks for the heads up Susann. Gremmie, I'll leave it to you to decide whether or not you wish to take this on or not for RN. Just let me know here or in a bug tracker issue.

Gremmie · Posted: Thu Jun 07, 2007 8:41 pm

I'm still a bit confused on where this is happening. The backend.php file is picking up the site slogan for the RSS feed? Susann, is the slogan saved in the nuke_config table as ü? Thanks.

Susann · Posted: Fri Jun 08, 2007 8:37 am

Yes, the backend.php shows it as description.
Gremmie just ad a word with ü or ä under preferences and there site slogan, check the db table nuke_config and check the backend.php. Go back change something under preferences, but not your word with ü, and check the database and backend.php again.
Let me know what you found in the db ü or ü.

bbuz · New Member Joined: May 17, 2006 Posts: 17

Im dealing with the same error

I use the latest ravennuke ver.

Cant use "ã" and "ç" in my website description. I got Rss feed errors when i use these characters..

cheers

Gremmie · Posted: Mon Jun 18, 2007 8:09 pm

Darn I still haven't looked into this. I am guessing that htmlentities() is being called before the data is saved in the config table. The backend.php script should probably call the "un-htmlentities()" on the data, and then XHTML'ize the data before presenting it in the feed. Too bad there isn't an xhtmlentities() function in PHP yet to turn all those characters into their decimal or hex equivalents.

Gremmie · Posted: Mon Jun 18, 2007 8:14 pm

Yeah, I just don't understand why the "named" entities like ü don't work, but the decimal equivalents (& # 252 ; )* do. Does XHTML have to include a special DTD to get the named entities? I will google this some more... Confused

* Note spaces were added to the decimal entity to get it to display in this post.

montego · Posted: Tue Jun 19, 2007 7:27 am

Gremmie, I think the issue is that we're not calling the entity_to_decimal_value() function in the backend.php script for every XML node value, for example, $sitename.

BTW, If you find a better "translator" of enties to decimal let me know as we should strongly consider making a generic function in mainfile.php or an include file for this that can be used on all RSS/XML feed scripts.

Gremmie · Posted: Tue Jun 19, 2007 5:28 pm

montego wrote:

Gremmie, I think the issue is that we're not calling the entity_to_decimal_value() function in the backend.php script for every XML node value, for example, $sitename.

Yes I should look at the code before I post. I agree, it is that easy.

I was confused on entities and XHTML. From what I was reading, I was hearing that XHTML did not accept named entities like ü but only their decimal values. But that is wrong. I was forgetting that the actual feed is in XML, and XML doesn't like the named entities...! Embarassed

As far as coming up with a better function for doing the mapping, it sure seems like PHP should provide something like that. But I don't think it does.

I'll add this to the bug tracker.

gregexp · Posted: Tue Jun 19, 2007 5:41 pm

ord() - Takes a character as an argument and returns the ASCII code in numerical format that corresponds to that character.
chr() - Takes an ASCII code, in numerical format, as an argument and retuns the character equivalent .

Another Solution could be to use the html_entities_decode() function which it seems it getting saved with htmlentities function, reversing it should help to get rid of the problem.

Unless I am completely off base, because backend.php uses htmlspecialchars() function all over the place in which I can only imagine that causing a problem.

Gremmie · Posted: Tue Jun 19, 2007 6:04 pm

Greg, no, we gotta take stuff like ü and turn it into & # 252; etc. These are non-ASCII characters.

html_entities_decode() would turn ü into ü and I'm not sure if that would validate.

If you look at backend.php like I should have done in the first place it is pretty obvious. There is a function to convert named entities into their decimal equivalents, and it needs to be applied to a few other things in the feed like $sitename, $topicname, etc.

gregexp · Posted: Tue Jun 19, 2007 9:31 pm

htmlspecialchars() was referenced in my last post, the reason I say this is because in backend.php its pulling data, already converted, there is a problem with that, if you have ü in your site title, then it gets coverted to ü then it gets outputed by backend.php using htmlspecialchars() again, it will turn into &uuml; and that is a problem, I'm not positive that is the problem creating your situation, but I feel that I might be on track. I dont see a reason for the htmlspecialchars function in backend.php if the admin file already converts it. From my reading, I have found no reason why ü would not work in the format. If I am mistaken, please let me know.

Gremmie · Posted: Wed Jun 20, 2007 7:29 am

Yes that could be it.

I don't think the data should be stored in the nuke_config table as ü . Perhaps ü would in fact be okay with XML. I'll have to test it out.

Susann · Posted: Fri Aug 03, 2007 5:37 pm

Gremmie Any news about your tests ?

Gremmie · Posted: Fri Aug 03, 2007 9:09 pm

I found that htmlentities() was being run on the data in admin/prefs before saving into the database, just as you observed Susann. I don't think we should be doing this. I could undo that and backend.php would work again, but I might break something else. Montego advised me to hold off on fixing this because someone else (kguske?) was close to getting a better rss feed working that would replace backend.php.

If you want to try removing the htmlentities() calls in admin/modules/settings.php right under the case 'ConfigSave': it might do the trick for you. But it may also break somewhere else. Montego added those, I believe, and he is on vacation right now. Maybe when he gets back he can chime in here.

Gremmie · Posted: Fri Aug 03, 2007 9:19 pm

And it looks like addslashes() needs to be added to more things there.... Sad

Edit (If we remove htmlentities, that is)