What encoding is used?

New Member Joined: Apr 29, 2013 Posts: 23

Sorry I couldn't come up with the best title but if a mod sees a more fitting one, please edit.

The question is this, I run gedit normally and for some reason I keep getting unrecognized characters warnings when opening php files such as rnconfig. By default it detects and tries to display the content as UTF-8.

Is that wrong?

What could be causing these warnings/errors, and could I possibly risk corruption?

So far I have been saving regardless of them and just making sure my backups are valid first, and I don't see any problems, even though I get another warning when saving. After I save I don't get the notices anymore, but I'd like to stop them in the first place.

Site Admin Joined: Aug 22, 2007 Posts: 1772

UTF-8 is not supported in the current version. I have it in use on my own pages but it was a massive work over weeks. Alone the Forums module was hard. I have it changed for some needed modules but not for all. For that you must change many many core-files of RN. You must also make sure to have the page and the dabase on the same encoding. A mix of both will not work.

Posted: Wed May 01, 2013 5:51 am

RN 2.51 is using ISO-8859-1. RN 3 will be UTF-8.

Posted: Wed May 01, 2013 8:10 am

This is strange. I expected that they are not utf-8, but now I suspect I was getting false readings. The notices aren't appearing anymore, and I did not hide them.

I also saved several files using utf-8, or so I am lead to believe, and they all seem to work fine without causing any problems with the site.

I will have to find something to better determine what the actual encoding of the files are, while I believe they are of the iso format, for some reason they don't all give the same notice with a pretty common app.

After adding the ISO-8859-1 encoding option to gedit and then re-opening something that was detected and then saved as UTF-8, I did not get the usual notice.

At this point I would have expected it to tell me it isn't ISO, but instead UTF-8. Perhaps I need to try a couple different ides, but why do these changes not seem to have any effect on nuke?

Posted: Wed May 01, 2013 8:45 am

The php-files can or should have utf-8 but the charset of the page should use ISO-8859-1. The database collation could use latin.

Posted: Wed May 01, 2013 7:54 pm

Thanks all.

If I open specifically rnconfig using UTF-8 I get the notice, but if I select ISO-8859-1 and re-load the file, I do not.

So it seems perhaps either some kind of encoding mismatch, or possibly certain files were saved with the iso encoding? Could this be a bug with rn?

Posted: Wed May 01, 2013 8:12 pm

I'm not sure if we are meaning the same but one question: How or better with what you are loading this file - with gedit as editor? Dreamweaver and Notpad++ says DOS\Windows or Westeuropean, that could create the notice in getid because Linux use uft-8 as default charset! The file was created on a windows machine.

Posted: Wed May 01, 2013 11:01 pm

While it does use default charset of UTF-8, gedit allows you to select the encoding when opening a file from the File>Open menu.

It could be gedit related, but it does not seem to be. It "seems" that some of the files are being saved differently, but it is ok. I just wanted to make sure I wouldn't harm anything by saving as a different encoding, and you said the files "can" be saved as UTF-8 so I assume that means it's OK if they are ISO instead.

I have seen no problems as of yet, but I have set the default encoding to ISO-8859-1 now, and nothing I open gives a notice, so either linux is confused and thinks they are actually ISO and NOT UTF-8, or theres a problem elsewhere.

I will do some testing with leafpad since it is a more common ide for devs, and I will also mention this in the #gnome as there are some folks there with a lot more knowledge on gedit than I do. If it does turn out to be a gedit bug I'll open a bug accordingly.

Posted: Thu May 02, 2013 2:44 am

There several people working on the files. Possibly one of these files created with a Windows setting. Wit 'can' i want say that is possible to use both.

The new editors are using all utf-8. Notpadd++ or Dreamweaver basically create all new files with UTF-8 in order to avoid such errors. I have exactly this problem when I create a txt-file using Windows and want to use it on Linux or mac os.

Create a new empty php file with your editor and copy the content of the affected file to the new. Save it under the same name and the error should disappear.

Posted: Thu May 02, 2013 5:00 pm

I don't believe switching back and forth between the two at the lower end. From Wikipedia:

Quote:

The first 128 characters of Unicode, which correspond one-to-one with ASCII, are encoded using a single octet with the same binary value as ASCII, making valid ASCII text valid UTF-8-encoded Unicode as well.

Which may be why you have not yet experienced any presentation issues. It would be a different story if you were to accidentally save the file with the BOM.

Posted: Thu May 02, 2013 8:42 pm

montego wrote:

I don't believe switching back and forth between the two at the lower end. From Wikipedia:

Quote:

The first 128 characters of Unicode, which correspond one-to-one with ASCII, are encoded using a single octet with the same binary value as ASCII, making valid ASCII text valid UTF-8-encoded Unicode as well.

Which may be why you have not yet experienced any presentation issues. It would be a different story if you were to accidentally save the file with the BOM.

Does that mean I can make the assumption that this would work:

neralex wrote:

Create a new empty php file with your editor and copy the content of the affected file to the new. Save it under the same name and the error should disappear.

So I can:

1) open the file.php with either utf-8/iso according to what i guess it is
2) select all and copy everything into an editor using utf-8 as default?
3) save the new file.php as utf-8 and start editing?

Posted: Thu May 02, 2013 9:11 pm

I believe the root of the issue may be certain characters in the file may trigger the editor to detect the file as UTF-8, even if it's not. There is one in rnconfig.php that probably shouldn't be there, even though it is in a comment. The copyright symbol for nukePIE.... © if you can somehow remove that and save properly, it should open in the correct encoding. There are likely other occurrences like this elsewhere, the ™ symbol (and other) would likely have the same effect in your editor; if save in an ansi file.