Thank you for posting this reference to the bug submission Miguel. I spoke with Bruce Armstrong at Elevate 2019 about an issue I was having with the FileOpen function not being able to interpret accents properly unless I coded it a certain way, which I found out through trial and error. He suggested I submit a bug and I was going to after I couldn't find an existing bug submission (as I was searching on FileOpen) but I never got around to it.



Hopefully the fix for FileEncoding will by default fix FileOpen but if not hopefully FileOpen will also be addressed.



Here is the code I used that ultimately worked:



li_file_handle = FileOpen(ls_path + This.Text(li_counter), StreamMode!, Read!, LockReadWrite!, Append!, EncodingANSI!) //EncodingANSI! required for accents

li_chars_read = FileReadEx(li_file_handle,lb_blob)

ls_total_file = String(lb_blob, EncodingUTF8!) //EncodingUTF8! required for accents



At the time this really struck me as odd having to open the file with ANSI encoding in order to properly read a UTF-8 encoded file, but doing so along with some conversion ended up working. I thought it might be related to byte order marking and very strict adherence to the absence or presence of BOM, but thanks to your bug submission I now know it is a bug.



Regards,

HI Mark,



I've the same problem and i've found the same workaround.



https://community.appeon.com/index.php/qna/q-a/fileencoding-utf8-file-returns-ansi



My problem is that the input files can be sometimes in ANSI, sometimes in UFT8.

The program doesn't know it.

So the conversion String(lb_blob, EncodingUTF8!) cannot always be done (if the file is ANSI, i get asian character).

So did you find a workaround to check the right file encoding ?

Hi Thierry...so for my current need at the moment I know in advance that the files I am receiving will be UTF-8 encoded (without BOM) and so I don't have the issue that you do. I am also automating this as the files come from an external source and so thankfully I don't have to see the files before processing them. If you/ your users aren't automating the process and are in a position to view the file before accepting it then you could write some code to preview it utilizing the different encoding (taking care when testing with UTF-8 due to the absence of BOM issue) and once it "looks" as it should, then finalize the process.



If not, then the only thing I can think of is to utilize Roland's suggestion re: IsTextUnicode but I thought in the other Q&A thread someone stated it didn't work or it didn't work enough maybe?!



Regards,