How to convert UTF-8 byte[] to string - Stack Overflow

You can easily convert the output string back to byte array by using Convert.FromBase64String . Note: The output string could contain '+', '/' and '='. If you ... Home Public Questions Tags Users Companies Collectives ExploreCollectives Teams StackOverflowforTeams –Startcollaboratingandsharingorganizationalknowledge. CreateafreeTeam WhyTeams? Teams CreatefreeTeam Collectives™onStackOverflow Findcentralized,trustedcontentandcollaboratearoundthetechnologiesyouusemost. LearnmoreaboutCollectives Teams Q&Aforwork Connectandshareknowledgewithinasinglelocationthatisstructuredandeasytosearch. LearnmoreaboutTeams HowtoconvertUTF-8byte[]tostring AskQuestion Asked 13years,4monthsago Modified 11monthsago Viewed 1.3mtimes 1098 Ihaveabyte[]arraythatisloadedfromafilethatIhappentoknowncontainsUTF-8. Insomedebuggingcode,Ineedtoconvertittoastring.Isthereaone-linerthatwilldothis? Underthecoversitshouldbejustanallocationandamemcopy,soevenifitisnotimplemented,itshouldbepossible. c#.netarraysstringtype-conversion Share Improvethisquestion Follow editedAug6,2021at16:10 PeterMortensen 30.6k2121goldbadges102102silverbadges124124bronzebadges askedJun16,2009at18:47 BCSBCS 72.9k6666goldbadges181181silverbadges291291bronzebadges 1 7 "shouldbejustanallocationandamemcopy":isnotcorrectbecausea.NETstringisUTF-16encoded.AUnicodecharactermightbeoneUTF-8codeunitoroneUTF-16codeunit.anothermightbetwoUTF-8codeunitsoroneUTF-16codeunit,anothermightbethreeUTF-8codeunitsoroneUTF-16codeunit,anothermightbefourUTF-8codeunitsortwoUTF-16codeunits.Amemcopymightbeabletowidenbutitwouldn'tbeabletohandleUTF-8toUTF-16conversion. – TomBlodget Nov19,2016at1:01 Addacomment  |  16Answers 16 Sortedby: Resettodefault Highestscore(default) Trending(recentvotescountmore) Datemodified(newestfirst) Datecreated(oldestfirst) 1692 stringresult=System.Text.Encoding.UTF8.GetString(byteArray); Share Improvethisanswer Follow editedFeb12,2015at20:19 JamesWebster 4,0062828silverbadges4141bronzebadges answeredJun16,2009at18:49 ZanoniZanoni 29.1k1212goldbadges5555silverbadges7272bronzebadges 9 18 howdoesithandlenullendedstrings? – maazza May12,2015at12:43 18 @maazzaforunknownreasonitdoesn'tatall.I'mcallingitlikeSystem.Text.Encoding.UTF8.GetString(buf).TrimEnd('\0');. – Hi-Angel Jul27,2015at7:53 20 @Hi-AngelUnknownreason?Theonlyreasonnull-terminatedstringseverbecamepopularwastheClanguage-andeventhatwasonlybecauseofahistoricaloddity(CPUinstructionsthatdealtwithnull-terminatedstrings)..NETonlyusesnull-terminatedstringswheninteroppingwithcodethatusesnull-terminatedstrings(whicharefinallydisappearing).It'sperfectlyvalidforastringtocontainNULcharacters.Andofcourse,whilenull-terminatedstringsaredeadsimpleinASCII(justbuilduntilyougetthefirstzerobyte),otherencodings,includingUTF-8,arenotsosimple. – Luaan Nov23,2015at10:05 4 OneofthebeautifulfeaturesofUTF-8isthatashortersequenceisneverasubsequenceofalongersequence.SoanullterminatedUTF-8stringissimple. – plugwash Nov24,2015at17:00 13 Well,goodluckunpackingitifithasnon-ascii.JustuseConvert.ToBase64String. – ErikBergstedt Dec12,2015at10:30  |  Show4morecomments 369 There'reatleastfourdifferentwaysdoingthisconversion. Encoding'sGetString,butyouwon'tbeabletogettheoriginalbytesbackifthosebyteshavenon-ASCIIcharacters. BitConverter.ToStringTheoutputisa"-"delimitedstring,butthere'sno.NETbuilt-inmethodtoconvertthestringbacktobytearray. Convert.ToBase64StringYoucaneasilyconverttheoutputstringbacktobytearraybyusingConvert.FromBase64String.Note:Theoutputstringcouldcontain'+','/'and'='.IfyouwanttousethestringinaURL,youneedtoexplicitlyencodeit. HttpServerUtility.UrlTokenEncodeYoucaneasilyconverttheoutputstringbacktobytearraybyusingHttpServerUtility.UrlTokenDecode.TheoutputstringisalreadyURLfriendly!ThedownsideisitneedsSystem.Webassemblyifyourprojectisnotawebproject. Afullexample: byte[]bytes={130,200,234,23};//Abytearraycontainsnon-ASCII(ornon-readable)characters strings1=Encoding.UTF8.GetString(bytes);//��� byte[]decBytes1=Encoding.UTF8.GetBytes(s1);//decBytes1.Length==10!! //decBytes1notsameasbytes //UsingUTF-8orotherEncodingobjectwillgetsimilarresults strings2=BitConverter.ToString(bytes);//82-C8-EA-17 String[]tempAry=s2.Split('-'); byte[]decBytes2=newbyte[tempAry.Length]; for(inti=0;iConvert.ToByte(ch,16)).ToArray(); – drtf Jul13,2014at14:43 Thisshouldbetheacceptedanswer.Itperfectlyillustratestheoutputofmultiplemethods.Thecurrentacceptedanswershowsonlyone,whichmaybeproblematicforsomedeveloperswhodon'tscrollthisfardown.-unlessyousortbyvotes,ofcourse. – dimitar.bogdanov Apr11,2021at8:36 Addacomment  |  33 Ageneralsolutiontoconvertfrombytearraytostringwhenyoudon'tknowtheencoding: staticstringBytesToStringConverted(byte[]bytes) { using(varstream=newMemoryStream(bytes)) { using(varstreamReader=newStreamReader(stream)) { returnstreamReader.ReadToEnd(); } } } Share Improvethisanswer Follow editedDec22,2015at14:31 slavoo 5,5786464goldbadges3636silverbadges3939bronzebadges answeredSep20,2015at8:24 NirNir 1,6582020silverbadges2020bronzebadges 1 5 ButthisassumesthatthereiseitheranencodingBOMinthebytestreamorthatitisinUTF-8.ButyoucandothesamewithEncodinganyway.Itdoesn'tmagicallysolvetheproblemwhenyoudon'tknowtheencoding. – SebastianZander Sep26,2017at17:05 Addacomment  |  13 Definition: publicstaticstringConvertByteToString(thisbyte[]source) { returnsource!=null?System.Text.Encoding.UTF8.GetString(source):null; } Using: stringresult=input.ConvertByteToString(); Share Improvethisanswer Follow editedAug1,2015at20:56 PeterMortensen 30.6k2121goldbadges102102silverbadges124124bronzebadges answeredOct16,2014at1:04 ErçinDedeoğluErçinDedeoğlu 4,49244goldbadges4444silverbadges6161bronzebadges 0 Addacomment  |  9 Convertingabyte[]toastringseemssimple,butanykindofencodingislikelytomessuptheoutputstring.Thislittlefunctionjustworkswithoutanyunexpectedresults: privatestringToString(byte[]bytes) { stringresponse=string.Empty; foreach(bytebinbytes) response+=(Char)b; returnresponse; } Share Improvethisanswer Follow editedAug6,2021at16:13 PeterMortensen 30.6k2121goldbadges102102silverbadges124124bronzebadges answeredApr22,2015at11:48 AndrewJEAndrewJE 84822goldbadges99silverbadges1919bronzebadges 2 IreceivedSystem.FormatExceptionusingyourmethodwhenIunpackeditwithConvert.FromBase64String. – ErikBergstedt Dec12,2015at10:20 @AndrewJEthiswilltakeforeventocomputeifyouhavealargebytearrayliketheoneusedfromthepictures. – user3841581 Nov4,2017at16:55 Addacomment  |  8 Isawsomeanswersatthispostandit'spossibletobeconsideredcompletedbaseknowledge,becauseIhaveaseveralapproachesinC#Programmingtoresolvethesameproblem.TheonlythingthatisnecessarytobeconsideredisaboutadifferencebetweenpureUTF-8andUTF-8withaBOM. Lastweek,atmyjob,IneededtodeveloponefunctionalitythatoutputsCSVfileswithaBOMandotherCSVfileswithpureUTF-8(withoutaBOM).EachCSVfileencodingtypewillbeconsumedbydifferentnon-standardizedAPIs.OneAPIreadsUTF-8withaBOMandtheotherAPIreadswithoutaBOM.Ineededtoresearchthereferencesaboutthisconcept,readingthe"What'sthedifferencebetweenUTF-8andUTF-8withoutBOM?"StackOverflowquestion,andtheWikipediaarticle"Byteordermark"tobuildmyapproach. Finally,myC#ProgrammingforbothUTF-8encodingtypes(withBOMandpure)neededtobesimilartothisexamplebelow: //ForUTF-8withBOM,equalssharedbyZanoni(attop) stringresult=System.Text.Encoding.UTF8.GetString(byteArray); //forPureUTF-8(withoutB.O.M.) stringresult=(newUTF8Encoding(false)).GetString(byteArray); Share Improvethisanswer Follow editedAug6,2021at16:34 PeterMortensen 30.6k2121goldbadges102102silverbadges124124bronzebadges answeredMay21,2020at21:26 AntonioLeonardoAntonioLeonardo 1,63611goldbadge88silverbadges1616bronzebadges 3 Don'tyouneedtospecificallystriptheBOMoffthestartthough?AsfarasIknow,evenifyouuseaUTF8EncodingwithBOM,itwillnotstripthatoffautomatically. – Nyerguds Jan14,2021at13:03 @Nyerguds,theUTF8Encodingobjectwith"false"valueatparameteriswithoutBOM. – AntonioLeonardo Feb12,2021at17:42 No,Imean,ifthetexthasaBOM,eventheSystem.Text.Encoding.UTF8willnotautomaticallystripthatoff.Tryitout. – Nyerguds Feb14,2021at1:41 Addacomment  |  7 Using(byte)b.ToString("x2"),Outputsb4b5dfe475e58b67 publicstaticclassExt{ publicstaticstringToHexString(thisbyte[]hex) { if(hex==null)returnnull; if(hex.Length==0)returnstring.Empty; vars=newStringBuilder(); foreach(bytebinhex){ s.Append(b.ToString("x2")); } returns.ToString(); } publicstaticbyte[]ToHexBytes(thisstringhex) { if(hex==null)returnnull; if(hex.Length==0)returnnewbyte[0]; intl=hex.Length/2; varb=newbyte[l]; for(inti=0;ix!=0)isaquickandeasywaytosolvethenullterminationproblem. – Nyerguds Sep21,2017at9:11 Whatdoyoumeanby"nulltermination"?Nullbytesintheinputarray?Canyoudefineexactlywhatyoumeaninyouranswer?(Butwithout"Edit:","Update:",orsimilar-theanswershouldappearasifitwaswrittentoday.) – PeterMortensen Aug6,2021at16:18 Idon'tfeeltheneedtoedittheanswer.Inlowlevelsystemsthatusebytearraysforascii-encodedstringsthearrayitselfdoesn'tcontaininformationaboutthelengthofthestring.Themostcommonpracticeistoterminatethestringwithavalueof0(akanull).Failingtodosoisthecauseofthefamousbufferoverflowexploit.Asforthisanswerspecifically,Ihaven'tusedc#inafewyearssoIdon'trememberifitjustwasn'tcopyingthenullbyteorfallingtostopcopyinguntilandincludingthenullbyte.Butthat'snullterminationinanutshell – Assimilater Aug7,2021at20:50 Ithinkmaybewhenitwascontinuingtocopypastthenullterminatorwithoutthiscodemaybe....butagainIdon'tremember – Assimilater Aug7,2021at20:55 Addacomment  |  2 ALINQone-linerforconvertingabytearraybyteArrFilenamereadfromafiletoapureASCIIC-stylezero-terminatedstringwouldbethis:Handyforreadingthingslikefileindextablesinoldarchiveformats. Stringfilename=newString(byteArrFilename.TakeWhile(x=>x!=0) .Select(x=>x<128?(Char)x:'?').ToArray()); Iuse'?'asthedefaultcharacterforanythingnotpureASCIIhere,butthatcanbechanged,ofcourse.Ifyouwanttobesureyoucandetectit,justuse'\0'instead,sincetheTakeWhileatthestartensuresthatastringbuiltthiswaycannotpossiblycontain'\0'valuesfromtheinputsource. Share Improvethisanswer Follow editedAug6,2021at16:14 PeterMortensen 30.6k2121goldbadges102102silverbadges124124bronzebadges answeredNov17,2016at8:12 NyergudsNyerguds 5,03211goldbadge3030silverbadges5959bronzebadges Addacomment  |  1 Trythisconsoleapplication: staticvoidMain(string[]args) { //Encoding_UTF8=Encoding.UTF8; string[]_mainString={"Hello,World!"}; Console.WriteLine("MainString:"+_mainString); //ConvertastringtoUTF-8bytes. byte[]_utf8Bytes=Encoding.UTF8.GetBytes(_mainString[0]); //ConvertUTF-8bytestoastring. string_stringuUnicode=Encoding.UTF8.GetString(_utf8Bytes); Console.WriteLine("StringUnicode:"+_stringuUnicode); } Share Improvethisanswer Follow editedAug6,2021at16:25 PeterMortensen 30.6k2121goldbadges102102silverbadges124124bronzebadges answeredSep29,2019at7:49 RMShahidulIslamShahedRMShahidulIslamShahed 1,1001212silverbadges1111bronzebadges Addacomment  |  0 Hereisaresultwhereyoudidn’thavetobotherwithencoding.Iuseditinmynetworkclassandsendbinaryobjectsasstringwithit. publicstaticbyte[]String2ByteArray(stringstr) { char[]chars=str.ToArray(); byte[]bytes=newbyte[chars.Length*2]; for(inti=0;iTheyaresupplementary? Wouldextractinghydrogenfromthesunlessenitslifespan? Whattranslation/versionoftheBiblewouldChaucerhaveread? Sub-zerocyclingwaterbottlesthatfitregularcages Traditionally,andcurrently,whatstopshumanvotecountersfromalteringballotstomakethem'Spoilt/Invalidvotes? WillIgetdeniedentryafterIremovedavisasticker?Ismypassportdamaged? Botchingcrosswindlandings Whois"Lord"in2Cor3:18 Myfavoriteanimalisa-singularandpluralform Sortbycolumngroupandignoreothercolumnsfailingforthisexample,why? Interpretinganegativeself-evaluationofahighperformer HowdoGPSreceiverscommunicatewithsatellites? PacifistethosblockingmyprogressinStellaris Doublelinemathsentence LaTeX2(e)vsLaTeX3 morehotquestions Questionfeed SubscribetoRSS Questionfeed TosubscribetothisRSSfeed,copyandpastethisURLintoyourRSSreader. lang-cs Yourprivacy Byclicking“Acceptallcookies”,youagreeStackExchangecanstorecookiesonyourdeviceanddiscloseinformationinaccordancewithourCookiePolicy. Acceptallcookies Customizesettings