How to decode a unicode string Python - Stack Overflow

You need to call encode function and not decode function, as item is already decoded. Like this: decoded_value = item.encode('utf-8'). Home Public Questions Tags Users Companies Collectives ExploreCollectives Teams StackOverflowforTeams –Startcollaboratingandsharingorganizationalknowledge. CreateafreeTeam WhyTeams? Teams CreatefreeTeam Collectives™onStackOverflow Findcentralized,trustedcontentandcollaboratearoundthetechnologiesyouusemost. LearnmoreaboutCollectives Teams Q&Aforwork Connectandshareknowledgewithinasinglelocationthatisstructuredandeasytosearch. LearnmoreaboutTeams HowtodecodeaunicodestringPython[duplicate] AskQuestion Asked 6years,8monthsago Modified 2years,6monthsago Viewed 62ktimes 12 Thisquestionalreadyhasanswershere: Whydoesn'tthisconversiontoutf8work? (4answers) Closed6yearsago. Whatisthebestwaytodecodeanencodedstringthatlookslike:u'u\xf1somestring'? Background:Ihavealistthatcontainsrandomvalues(stringsandintegers),I'mtryingtoconverteveryiteminthelisttoastringthenprocesseachofthem. Turnsoutsomeoftheitemsareoftheformat:u'u\xf1somestring' WhenItriedconvertingtoastring,Igettheerror:UnicodeEncodeError:'ascii'codeccan'tencodecharacteru'\xf1'inposition1:ordinalnotinrange(128) Ihavetried item=u'u\xf1somestring' decoded_value=item.decode('utf-8','ignore') However,Ikeepgettingthesameerror. IhavereadupaboutunicodecharactersandtriedanumberofsuggestionsfromSObutnonehaveworkedsofar.AmImissingsomethinghere? stringpython-2.7unicodedecodeencode Share Improvethisquestion Follow editedMar22,2020at16:54 mfalade askedJan29,2016at11:26 mfalademfalade 1,11711goldbadge1616silverbadges1616bronzebadges 4 Ifit'saUnicodestring,it'salreadydecoded. – RemcoGerlich Jan29,2016at11:31 Youmayfindthisarticlehelpful:PragmaticUnicode,whichwaswrittenbySOveteranNedBatchelder. – PM2Ring Jan29,2016at11:32 Iassumeyou'reusingPython2.YoushouldalwaysmentionthePythonversionwithUnicodequestions(preferablywiththeappropriatetag)becausePython2&Python3handleUnicoderatherdifferently. – PM2Ring Jan29,2016at11:33 FWIW,s=u'u\xf1somestring'.encode('utf-8');prints,repr(s)printsuñsomestring'u\xc3\xb1somestring' – PM2Ring Jan29,2016at11:37 Addacomment  |  2Answers 2 Sortedby: Resettodefault Highestscore(default) Trending(recentvotescountmore) Datemodified(newestfirst) Datecreated(oldestfirst) 15 Youneedtocallencodefunctionandnotdecodefunction,asitemisalreadydecoded. Likethis: decoded_value=item.encode('utf-8') Share Improvethisanswer Follow editedJan30,2016at7:29 answeredJan29,2016at11:30 SameerMirjiSameerMirji 2,0731515silverbadges2828bronzebadges 3 1 YoudecodetoUnicode,encodetobytestrings. – MarkTolonen Jan29,2016at17:01 @MarkTolonen:Sowhatpartofmyanswerdidyoufindwronghere?I'vespecificallyusedcodeblockstoindicateIwastakingaboutthemethodnameshere. – SameerMirji Jan30,2016at4:24 1 ThestringisalreadydecodedifitisaUnicodestring.item.encode('utf-8')makesanencoded_value.You(andtheOP)havetheterminologyconfused. – MarkTolonen Jan30,2016at7:27 Addacomment  |  3 Thatstringalreadyisdecoded(it'saUnicodeobject).Youneedtoencodeitifyouwanttostoreitinafile(orsendittoadumbterminaletc.). Generally,whenworkingwithUnicode,youshould(inPython2)decodeallyourstringsearlyintheworkflow(whichyoualreadyseemtohavedone;manylibrariesthathandleinternettrafficwillalreadydothatforyou),thendoallyourworkonUnicodeobjects,andthenattheveryend,whenwritingthemback,encodethemtowhateverencodingyou'reusing. Share Improvethisanswer Follow answeredJan29,2016at11:31 TimPietzckerTimPietzcker 318k5656goldbadges494494silverbadges550550bronzebadges Addacomment  |  Nottheansweryou'relookingfor?Browseotherquestionstaggedstringpython-2.7unicodedecodeencodeoraskyourownquestion. TheOverflowBlog HowtoearnamillionreputationonStackOverflow:beofservicetoothers Therightwaytojobhop(Ep.495) FeaturedonMeta BookmarkshaveevolvedintoSaves Inboximprovements:markingnotificationsasread/unread,andafiltered... Revieweroverboard!Orarequesttoimprovetheonboardingguidancefornew... CollectivesUpdate:RecognizedMembers,Articles,andGitLab Shouldweburninatethe[script]tag? Linked 14 Whydoesn'tthisconversiontoutf8work? 0 Pythonprintsstringwith"+"signinbetween 0 CharacterDecodingencodedstringbacktooriginalstatepython Related 7319 WhatisthedifferencebetweenStringandstringinC#? 3288 HowdoIiterateoverthewordsofastring? 4567 HowdoIread/convertanInputStreamintoaStringinJava? 3469 Convertbytestoastring 2557 HowdoIgetasubstringofastringinPython? 4802 HowdoImakethefirstletterofastringuppercaseinJavaScript? 5276 HowdoIreplacealloccurrencesofastringinJavaScript? 7412 HowtocheckwhetherastringcontainsasubstringinJavaScript? 3588 DoesPythonhaveastring'contains'substringmethod? 2455 HowdoIlowercaseastringinPython? HotNetworkQuestions Theunusualphrasing"verb+the+comparativeadjective"intheLordoftheRingsnovels HowtogetridofUbuntuProadvertisementwhenupdatingapt? HowcanIkeepmyampfromtemperingthetoneofmyprocessor?(rockandhardmetalmusic) Howdouncomputablenumbersrelatetouncomputablefunctions? Myfavoriteanimalisa-singularandpluralform Wouldmerfolkgainanyrealadvantagefrommounts(andbeastsofburden)? Whydoesn'ttheMBRS1100SchottkydiodehaveanexponentialI/Vcharacteristic? ShouldIresendanapplication? WhatistheAmericanequivalentof"Icalledmymomtoaskafterher"? UnderstandingsuspiciousHTTPGETRequest WhatdothecolorsindicateonthisKC135tankerboom? Canyoufindit? Whatprotocolisthiswaveform? Realitycheck:PolarCO2lakescoexistingwithanequatorialH2Oocean Unsurewhatthesewatersoftenerdialsarefor Adecimal-basedunitoftime MakeaCourtTranscriber Whydoes«facture»mean"bill,invoice"? Movingframesmethod PreferenceofBJTtoMOSFET Howtoplug2.5mm²strandedwiresintoapushwirewago? Canaphotonturnaprotonintoaneutron? Whataretheargumentsforrevengeandretribution? Wouldextractinghydrogenfromthesunlessenitslifespan? morehotquestions lang-py Yourprivacy Byclicking“Acceptallcookies”,youagreeStackExchangecanstorecookiesonyourdeviceanddiscloseinformationinaccordancewithourCookiePolicy. Acceptallcookies Customizesettings