Wrong ANSI encoding for UTF-8 text file, when "Auto-detect ...

Even if I click in Notepad++ "Encoding - Convert to UTF-8" and save the file, then reopening will show that the file is still ANSI. Or I don't ... Skiptocontent {{message}} notepad-plus-plus / notepad-plus-plus Public Notifications Fork 3.8k Star 17.2k Code Issues 1.7k Pullrequests 19 Actions Projects 0 Wiki Security Insights More Code Issues Pullrequests Actions Projects Wiki Security Insights Newissue Haveaquestionaboutthisproject?SignupforafreeGitHubaccounttoopenanissueandcontactitsmaintainersandthecommunity. Pickausername EmailAddress Password SignupforGitHub Byclicking“SignupforGitHub”,youagreetoourtermsofserviceand privacystatement.We’lloccasionallysendyouaccountrelatedemails. AlreadyonGitHub? Signin toyouraccount Jumptobottom WrongANSIencodingforUTF-8textfile,when"Auto-detectcharactersencoding"isdisabled. #11667 Open TIMONz1535openedthisissue May13,2022 ·12comments Open WrongANSIencodingforUTF-8textfile,when"Auto-detectcharactersencoding"isdisabled. #11667 TIMONz1535openedthisissue May13,2022 ·12comments Comments Copylink TIMONz1535 commented May13,2022 • edited DescriptionoftheIssue Notepad++opensasometextfilealwaysinANSIencoding,despitethefactthatthefileitselfisUTF-8and"Auto-detectcharactersencoding"isdisabled.Notreproducedwithothermylogfiles. wrong_encoding_npp.txt Ireplacedallthecharactersinthelogforprivacyreasonswiththeirequivalents.ThelogcontainsRussiancharactersandsomeUTF-8symbolsthattheplayerusedinhisnickname"『𝕊』『ℂ』『ℙ』". StepstoReproducetheIssue Disable"Auto-detectcharactersencoding"(thirdcheckboxinMisc,right?IhaveRussianlanguagepack) OpenUTF-8logfile(asplaintext) GotANSIencoding ExpectedBehavior Openthefileintheencodinginwhichitiswritten,withoutanymagic.UTF-8encodingforUTF-8textfile. IpreferthatallfilesareconsideredasUTF-8bydefault,consideringthattheauto-detectoptionisdisabled. ActualBehavior ANSIencodingforthisUTF-8textfile. DebugInformation Notepad++v8.3.3(32-bit) Buildtime:Mar132022-17:11:10 Path:C:\ProgramFiles(x86)\Notepad++\notepad++.exe CommandLine:"C:\Users\Drakon4ik\Desktop\Desktop\wrong_encoding_npp.txt" Adminmode:OFF LocalConfmode:OFF CloudConfig:OFF OSName:Windows10Enterprise(64-bit) OSVersion:2009 OSBuild:19044.1706 CurrentANSIcodepage:65001 Plugins:ComparePlugin.dllDSpellCheck.dllmimeTools.dllNppConverter.dllNppExport.dllNppTextFX.dll Windows10Pro21H2(19044.1706) AllSettings-Time&Language-Language-"AdministrativeLanguageSettings"-"Beta:UseUnicodeUTF-8forworldwidelanguagesupport"isenabled NppLanguagepack:Russian Preferences-Misc-"Auto-detectcharactersencoding"isdisabled(thirdcheckbox). Newdocument-UTF-8isselected,andchekcbox"ApplytoopenedANSIfiles"alsoenabled. Thetextwasupdatedsuccessfully,buttheseerrorswereencountered: Allreactions Copylink rimrul commented May13,2022 So,letmegetthisstraight.Youdisable"autodetectfileencoding"andexpectnotepad++todetectthefileencoding? Allreactions Sorry,somethingwentwrong. Copylink Author TIMONz1535 commented May13,2022 • edited No,IcreatedafilewithUTF-8encoding IopenitinNotepad++anditshowsasANSI EvenifIclickinNotepad++"Encoding-ConverttoUTF-8"andsavethefile,thenreopeningwillshowthatthefileisstillANSI. OrIdon'tunderstandwhatthe"Auto-detectcharactersencoding"optionmeans,ifIdisabledit,inwhatencodingshouldthefilebeopened?Howcantheappdeterminetheencodingofafilewithoutthisoption?Whatisthedefaultencoding? Thatis,frommyside,Iwouldjustsay"alwaysthinkthatallfilesareinUTF-8",canthisbedone? Bytheway,settingordisabling"Auto-detectcharactersencoding"optiondoesnotaffectthesamplefile,anddoesnotchangethebehavioronmyotherlogs. Allreactions Sorry,somethingwentwrong. Copylink Contributor rdipardo commented May14,2022 ..."alwaysthinkthatallfilesareinUTF-8",canthisbedone? Windowsnotepaddoesthatbydefault: Allreactions Sorry,somethingwentwrong. Copylink ValZapod commented Jun11,2022 ANSIIencodingcannotbewrongforUTF-8,itisbackwardcompatiblewithASCII.RememberthatASCIIisonlyhalfof8bitsuntil7FandUTF-8morethan7Fisimpossiblefor1byte. Allreactions Sorry,somethingwentwrong. Copylink alankilborn commented Jun11,2022 @rdipardoDoyouhaveideasforchangingNotepad++toactuallyopenthisfileasUTF-8bydefaultinthedescribedsituation? Allreactions Sorry,somethingwentwrong. Copylink Contributor rdipardo commented Jun13,2022 Iwouldfirstofallask@TIMONz1535if"Apply[UTF-8encoding]toopenANSIfiles"isselectedatSettings>Preferences>NewDocument: Allreactions Sorry,somethingwentwrong. Copylink alankilborn commented Jun15,2022 if"Apply[UTF-8encoding]toopenANSIfiles"isselected Myunderstandingofthatsettingisthatifitistickeditwill: openempty(zero-length)filesasUTF-8insteadofANSI openpurely7-bitdatafilesasUTF-8insteadofANSI andnothingmore. IntheOP'scase,hehasneithercondition. Ofcoursemyunderstandingofhowthatcheckboxworkscouldbedeficient... Allreactions Sorry,somethingwentwrong. Copylink ValZapod commented Jun15,2022 openpurely7-bitdatafilesasUTF-8insteadofANSI Youmean7bitin8bits.7bitisonlysupportedinSMS. Allreactions Sorry,somethingwentwrong. Copylink alankilborn commented Jun15,2022 • edited WhenIsay"purely7-bitdata"Imeanthatnobyteinthefilehasthedecimalvalue128to255,inclusive. Allreactions Sorry,somethingwentwrong. Copylink Author TIMONz1535 commented Jun19,2022 • edited Hi,Itesteditonthenewversionandthebehavioristhesame. Notepad++v8.4.2(32-bit) Buildtime:May292022-16:45:17 Path:C:\ProgramFiles(x86)\Notepad++\notepad++.exe CommandLine:"C:\Users\Shararvev\Downloads\wrong_encoding_npp.txt" Adminmode:OFF LocalConfmode:OFF CloudConfig:OFF OSName:Windows10Enterprise(64-bit) OSVersion:21H2 OSBuild:19044.1766 CurrentANSIcodepage:65001 Plugins: ComparePlugin(2.0.2) DSpellCheck(1.4.24) mimeTools(2.8) NppConverter(4.4) NppExport(0.4) NppTextFX(0.2.6) Asitwaswritteninthebodyoftheissue,ApplytoopenedANSIfilesisenabled. Settings>Preferences>NewDocument>UTF-8isselected,andchekcbox"ApplytoopenedANSIfiles"alsoenabled. Preferences-Misc-"Auto-detectcharactersencoding"isdisabled. WhenIturnedAuto-detectbackon,nothingchanged.Theattachedfilealwaysopensas"ANSI". Thenewemptyfileisdisplayedas"UTF-8". Bytheway,ifyouhaven'tlookedattheattachedfile,youmayfindthatdeletingafewlinescausestheNPPtoseethisfileas"UTF-8".Thismagicalbehavior,ontopofeverythingelse,mademecreatethisissue(butIforgottowriteitrightaway😅). Allreactions Sorry,somethingwentwrong. Copylink alankilborn commented Jun19,2022 youmayfindthatdeletingafewlinescausestheNPPtoseethisfileas"UTF-8" Itmighthelpthedevsforyoutocomeupwithadefinitecaseforthis,forexample,"openfile,seeANSI,deletelinesXX-YY,save,close,openfile,seeUTF-8". 😕 1 TIMONz1535reactedwithconfusedemoji Allreactions 😕 1reaction Sorry,somethingwentwrong. Copylink Contributor rdipardo commented Jun19,2022 I'venoticedthatbothof@TIMONz1535'sDebugInfosindicatethe"CurrentANSIcodepage"isUTF-8(65001)(*).Thismightcreatetheexpectationthateverydocumentwillinheritthatsetting.Infact,Notepad++getsthatvaluedirectlyfromtheoperatingsystemviatheGetACP()Win32APIfunction: notepad-plus-plus/PowerEditor/src/WinControls/AboutDlg/AboutDlg.cpp Line287 in 3b04793 generic_sprintf(szACP,TEXT("%u"),::GetACP()); Ithasnothingtodowiththeencodingoftheactivedocument.Thedocument'scodepageissetbytheeditorandcanbequeriedbysendingtheSCI_GETCODEPAGEScintillamessage. (*)SettingWindowstouseUTF-8everywhereisarelativelynewfeature,andthere'snothinginN++'scodetosuggestthedevelopersareevenawareitexists. Press⊞(Windowskey)+R Enterintl.cpl Clickthe"Administrative"tab Click"Changesystemlocale..." Check"Beta:UseUnicodeUTF-8forworldwidelanguagesupport"andreboot Allreactions Sorry,somethingwentwrong. Signupforfree tojointhisconversationonGitHub. Alreadyhaveanaccount? Signintocomment Assignees Nooneassigned Labels Noneyet Projects Noneyet Milestone Nomilestone Development Nobranchesorpullrequests 5participants Youcan’tperformthatactionatthistime. Yousignedinwithanothertaborwindow.Reloadtorefreshyoursession. Yousignedoutinanothertaborwindow.Reloadtorefreshyoursession.