utf-16le[BOM] to utf-8 file solution - GitHub

http://stackoverflow.com/questions/22459020/python-decode-utf-16-file-with-bom. import codecs. encoded_text = open('utf16lebom_file', 'rb').read() #you ... Skiptocontent Allgists BacktoGitHub Signin Signup Sign in Sign up {{message}} Instantlysharecode,notes,andsnippets. ctjoy/utf16leBOM_to_utf8.py LastactiveMar25,2018 Star 0 Fork 0 Star Code Revisions 2 Embed Whatwouldyouliketodo? Embed Embedthisgistinyourwebsite. Share Copysharablelinkforthisgist. Clonevia HTTPS ClonewithGitorcheckoutwithSVNusingtherepository’swebaddress. LearnmoreaboutcloneURLs DownloadZIP utf-16le[BOM]toutf-8filesolution Raw utf16leBOM_to_utf8.py ThisfilecontainsbidirectionalUnicodetextthatmaybeinterpretedorcompileddifferentlythanwhatappearsbelow.Toreview,openthefileinaneditorthatrevealshiddenUnicodecharacters. LearnmoreaboutbidirectionalUnicodecharacters Showhiddencharacters #http://stackoverflow.com/questions/22459020/python-decode-utf-16-file-with-bom importcodecs encoded_text=open('utf16lebom_file','rb').read()#youshouldreadinbinarymodetogettheBOMcorrectly bom=codecs.BOM_UTF16_LE#printdir(codecs)forotherencodings assertencoded_text.startswith(bom)#makesuretheencodingiswhatyouexpect,otherwiseyou'llgetwrongdata encoded_text=encoded_text[len(bom):]#stripawaytheBOM decoded_text=encoded_text.decode('utf-16le') f=open('utf8_file','wb') f.write(decoded_text.encode('utf8')) f.close() Signupforfree tojointhisconversationonGitHub. Alreadyhaveanaccount? Signintocomment Youcan’tperformthatactionatthistime. Yousignedinwithanothertaborwindow.Reloadtorefreshyoursession. Yousignedoutinanothertaborwindow.Reloadtorefreshyoursession.