There is always a current working directory, whether we're in the Python Shell, ... coding: utf-8 -*- >>> file = open('Alone.txt', encoding='utf-8') ...
Togglenavigation
BogoToBogo
Home
About
BigData
MachineLearning
AngularJS
Python
C++
go
DevOps
Kubernetes
Algorithms
More...
Qt5
Android
Linux
Java
CSharp
VideoStreaming
FFmpeg
Matlab
Django1.8
Laravel5.2
RubyOnRails
HTML5&CSS
PythonFilesandos.path
bogotobogo.comsitesearch:
Directories
Themodulecalledoscontainsfunctionstogetinformationonlocaldirectories,files,processes,andenvironmentvariables.
os.getcwd()
ThecurrentworkingdirectoryisapropertythatPythonholdsinmemoryatalltimes.Thereisalwaysacurrentworkingdirectory,whetherwe'reinthePythonShell,runningourownPythonscriptfromthecommandline,etc.
>>>importos
>>>print(os.getcwd())
C:\Python32
>>>os.chdir('/test')
>>>print(os.getcwd())
C:\test
Weusedtheos.getcwd()functiontogetthecurrentworkingdirectory.WhenwerunthegraphicalPythonShell,thecurrentworkingdirectorystartsasthedirectorywherethePythonShellexecutableis.OnWindows,thisdependsonwhereweinstalledPython;thedefaultdirectoryisc:\Python32.IfwerunthePythonShellfromthecommandline,thecurrentworkingdirectorystartsasthedirectorywewereinwhenweranpython3.
Then,weusedtheos.chdir()functiontochangethecurrentworkingdirectory.
Notethatwhenwecalledtheos.chdir()function,weusedaLinux-stylepathname(forwardslashes,nodriveletter)eventhoughwe'reonWindows.ThisisoneoftheplaceswherePythontriestopaperoverthedifferencesbetweenoperatingsystems.
os.path.join()
os.pathcontainsfunctionsformanipulatingfilenamesanddirectorynames.
>>>importos
>>>print(os.path.join('/test/','myfile'))
/test/myfile
>>>print(os.path.expanduser('~'))
C:\Users\K
>>>print(os.path.join(os.path.expanduser('~'),'dir','subdir','k.py'))
C:\Users\K\dir\subdir\k.py
Theos.path.join()functionconstructsapathnameoutofoneormorepartialpathnames.Inthiscase,itsimplyconcatenatesstrings.Callingtheos.path.join()functionwilladdanextraslashtothepathnamebeforejoiningittothefilename.
Theos.path.expanduser()functionwillexpandapathnamethatuses~torepresentthecurrentuser'shomedirectory.Thisworksonanyplatformwhereusershaveahomedirectory,includingLinux,MacOSX,andWindows.Thereturnedpathdoesnothaveatrailingslash,buttheos.path.join()functiondoesn'tmind.
Combiningthesetechniques,wecaneasilyconstructpathnamesfordirectoriesandfilesintheuser'shomedirectory.Theos.path.join()functioncantakeanynumberofarguments.
Note:weneedtobecarefulaboutthestringwhenweuseos.path.join.Ifweuse"/",ittellsPythonthatwe'reusingabsolutepath,anditoverridesthepathbeforeit:
>>>importos
>>>print(os.path.join('/test/','/myfile'))
/myfile
Aswecanseethepath"/test/"isgone!
os.path.split()
os.pathalsocontainsfunctionstosplitfullpathnames,directorynames,andfilenamesintotheirconstituentparts.
>>>pathname="/Users/K/dir/subdir/k.py"
>>>os.path.split(pathname)
('/Users/K/dir/subdir','k.py')
>>>(dirname,filename)=os.path.split(pathname)
>>>dirname
'/Users/K/dir/subdir'
>>>pathname
'/Users/K/dir/subdir/k.py'
>>>filename
'k.py'
>>>(shortname,extension)=os.path.splitext(filename)
>>>shortname
'k'
>>>extension
'.py'
Thesplit()functionsplitsafullpathnameandreturnsatuplecontainingthepathandfilename.Theos.path.split()functiondoesreturnmultiplevalues.Weassignthereturnvalueofthesplitfunctionintoatupleoftwovariables.Eachvariablereceivesthevalueofthecorrespondingelementofthereturnedtuple.Thefirstvariable,dirname,receivesthevalueofthefirstelementofthetuplereturnedfromtheos.path.split()function,thefilepath.Thesecondvariable,filename,receivesthevalueofthesecondelementofthetuplereturnedfromtheos.path.split()function,thefilename.
os.pathalsocontainstheos.path.splitext()function,whichsplitsafilenameandreturnsatuplecontainingthefilenameandthefileextension.Weusedthesametechniquetoassigneachofthemtoseparatevariables.
glob.glob()
TheglobmoduleisanothertoolinthePythonstandardlibrary.It'saneasywaytogetthecontentsofadirectoryprogrammatically,anditusesthesortofwildcardsthatwemayalreadybefamiliarwithfromworkingonthecommandline.
>>>importglob
>>>os.chdir('/test')
>>>importglob
>>>glob.glob('subdir/*.py')
['subdir\\tes3.py','subdir\\test1.py','subdir\\test2.py']
Theglobmoduletakesawildcardandreturnsthepathofallfilesanddirectoriesmatchingthewildcard.
Filemetadata
Everyfilesystemstoresmetadataabouteachfile:creationdate,last-modifieddate,filesize,andsoon.PythonprovidesasingleAPItoaccessthismetadata.Wedon'tneedtoopenthefileandallweneedisthefilename.
>>>importos
>>>print(os.getcwd())
C:\test
>>>os.chdir('subdir')
>>>print(os.getcwd())
C:\test\subdir
>>>metadata=os.stat('test1.py')
>>>metadata.st_mtime
1359868355.9555483
>>>importtime
>>>time.localtime(metadata.st_mtime)
time.struct_time(tm_year=2013,tm_mon=2,tm_mday=2,tm_hour=21,
tm_min=12,tm_sec=35,tm_wday=5,tm_yday=33,tm_isdst=0)
>>>metadata.st_size
1844
Callingtheos.stat()functionreturnsanobjectthatcontainsseveraldifferenttypesofmetadataaboutthefile.st_mtimeisthemodificationtime,butit'sinaformatthatisn'tterriblyuseful.Actually,it'sthenumberofsecondssincetheEpoch,whichisdefinedasthefirstsecondofJanuary1st,1970.
ThetimemoduleispartofthePythonstandardlibrary.Itcontainsfunctionstoconvertbetweendifferenttimerepresentations,formattimevaluesintostrings,andfiddlewithtimezones.
Thetime.localtime()functionconvertsatimevaluefromseconds-since-the-Epoch(fromthest_mtimepropertyreturnedfromtheos.stat()function)intoamoreusefulstructureofyear,month,day,hour,minute,second,andsoon.ThisfilewaslastmodifiedonFeb2,2013,ataround9:12PM.
Theos.stat()functionalsoreturnsthesizeofafile,inthest_sizeproperty.Thefile"test1.py"is1844bytes.
os.path.realpath()-Absolutepathname
Theglob.glob()functionreturnedalistofrelativepathnames.Ifweuwanttoconstructanabsolutepathname-i.e.onethatincludesallthedirectorynamesbacktotherootdirectoryordriveletter-thenwe'llneedtheos.path.realpath()function.
>>>importos
>>>print(os.getcwd())
C:\test\subdir
>>>print(os.path.realpath('test1.py'))
C:\test\subdir\test1.py
os.path.expandvars()-Env.variable
Theexpandvarsfunctioninsertsenvironmentvariablesintoafilename.
>>>importos
>>>os.environ['SUBDIR']='subdir'
>>>print(os.path.expandvars('/home/users/K/$SUBDIR'))
/home/users/K/subdir
OpeningFiles
Toopenafile,weusebuilt-inopen()function:
myfile=open('mydir/myfile.txt','w')
Theopen()functiontakesafilenameasanargument.Herethefilenameismydir/myfile.txt,andthenextargumentisaprocessingmode.Themodeisusuallythestring'r'toopentextinput(thisisthedefaultmode),'w'tocreateandopenopenfortextoutput.Thestring'a'istoopenforappendingtexttotheend.Themodeargumentcanspecifyadditionaloptions:addinga'b'tothemodestringallowsforbinarydata,andaddinga+opensthefileforbothinputandoutput.
Thetablebelowlistsseveralcombinationoftheprocessingmodes:
Mode
Description
r
Opensafileforreadingonly.Thefilepointerisplacedatthebeginningofthefile.Thisisthedefaultmode.
rb
Opensafileforreadingonlyinbinaryformat.Thefilepointerisplacedatthebeginningofthefile.Thisisthedefaultmode.
r+
Opensafileforbothreadingandwriting.Thefilepointerwillbeatthebeginningofthefile.
rb+
Opensafileforbothreadingandwritinginbinaryformat.Thefilepointerwillbeatthebeginningofthefile.
w
Opensafileforwritingonly.Overwritesthefileifthefileexists.Ifthefiledoesnotexist,createsanewfileforwriting.
wb
Opensafileforwritingonlyinbinaryformat.Overwritesthefileifthefileexists.Ifthefiledoesnotexist,createsanewfileforwriting.
w+
Opensafileforbothwritingandreading.Overwritestheexistingfileifthefileexists.Ifthefiledoesnotexist,createsanewfileforreadingandwriting.
a
Opensafileforappending.Thefilepointerisattheendofthefileifthefileexists.Thatis,thefileisintheappendmode.Ifthefiledoesnotexist,itcreatesanewfileforwriting.
ab
Opensafileforappendinginbinaryformat.Thefilepointerisattheendofthefileifthefileexists.Thatis,thefileisintheappendmode.Ifthefiledoesnotexist,itcreatesanewfileforwriting.
a+
Opensafileforbothappendingandreading.Thefilepointerisattheendofthefileifthefileexists.Thefileopensintheappendmode.Ifthefiledoesnotexist,itcreatesanewfileforreadingandwriting.
ab+
Opensafileforbothappendingandreadinginbinaryformat.Thefilepointerisattheendofthefileifthefileexists.Thefileopensintheappendmode.Ifthefiledoesnotexist,itcreatesanewfileforreadingandwriting.
Therearethingsweshouldknowaboutthefilename:
It'snotjustthenameofafile.It'sacombinationofadirectorypathandafilename.InPython,wheneverweneedafilename,wecanincludesomeorallofadirectorypathaswell.
Thedirectorypathusesaforwardslashwithoutmentioningoperatingsystem.Windowsusesbackwardslashestodenotesubdirectories,whileLinuxuseforwardslashes.ButinPython,forwardslashesalwayswork,evenonWindows.
Thedirectorypathdoesnotbeginwithaslashoradriveletter,soitiscalledarelativepath.
It'sastring.AllmodernoperatingsystemsuseUnicodetostorethenamesoffilesanddirectories.Python3fullysupportsnon-ASCIIpathnames.
CharacterEncoding
AstringisasequenceofUnicodecharacters.AfileondiskisnotasequenceofUnicodecharactersbutratherasequenceofbytes.Soifwereadafilefromdisk,howdoesPythonconvertthatsequenceofbytesintoasequenceofcharacters?Internally,PythondecodesthebytesaccordingtoaspecificcharacterencodingalgorithmandreturnsasequenceofUnicodecharacterstring.
Ihaveafile('Alone.txt'):
나혼자(Alone)-BySistar
추억이이리많을까넌대체뭐할까
아직난이래혹시돌아올까봐
Let'strytoreadthefile:
>>>file=open('Alone.txt')
>>>str=file.read()
Traceback(mostrecentcalllast):
File"",line1,in
str=file.read()
File"C:\Python32\lib\encodings\cp1252.py",line23,indecode
returncodecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError:'charmap'codeccan'tdecodebyte0x90inposition6:charactermapsto
>>>
Whatjusthappened?
Wedidn'tspecifyacharacterencoding,soPythonisforcedtousethedefaultencoding.
What'sthedefaultencoding?Ifwelookcloselyatthetraceback,wecanseethatit'scrashingincp1252.py,meaningthatPythonisusingCP-1252asthedefaultencodinghere.(CP-1252isacommonencodingoncomputersrunningMicrosoftWindows.)TheCP-1252charactersetdoesn'tsupportthecharactersthatareinthisfile,sothereadfailswithanUnicodeDecodeError.
Actually,whenIdisplaytheKoreancharacter,Ihadtoputthefollowinglinesofhtmltotheheadersection:
ASCIIandUnicode
Therearecharacterencodingsforeachmajorlanguageintheworld.Sinceeachlanguageisdifferent,andmemoryanddiskspacehavehistoricallybeenexpensive,eachcharacterencodingisoptimizedforaparticularlanguage.Eachencodingusingthesamenumbers(0-255)torepresentthatlanguage'scharacters.Forinstance,theASCIIencoding,whichstoresEnglishcharactersasnumbersrangingfrom0to127.(65iscapitalA,97islowercasea).Englishhasaverysimplealphabet,soitcanbecompletelyexpressedinlessthan128numbers.
WesternEuropeanlanguageslikeFrench,Spanish,andGermanhavemorelettersthanEnglish.ThemostcommonencodingfortheselanguagesisCP-1252.TheCP-1252encodingsharescharacterswithASCIIinthe0-127range,butthenextendsintothe128-255rangeforcharacterslikeñ,ü,etc.It'sstillasingle-byteencoding,though;thehighestpossiblenumber,255,stillfitsinonebyte.
ThentherearelanguageslikeChineseandKorean,whichhavesomanycharactersthattheyrequiremultiple-bytecharactersets.Thatis,eachcharacterisrepresentedbyatwo-bytenumber(0-65535).Butdifferentmulti-byteencodingsstillsharethesameproblemasdifferentsingle-byteencodings,namelythattheyeachusethesamenumberstomeandifferentthings.It'sjustthattherangeofnumbersisbroader,becausetherearemanymorecharacterstorepresent.
Unicodeisdesignedtorepresenteverycharacterfromeverylanguage.Unicoderepresentseachletter,character,orideographasa4-bytenumber.Eachnumberrepresentsauniquecharacterusedinatleastoneoftheworld'slanguages.Thereisexactly1numberpercharacter,andexactly1characterpernumber.Everynumberalwaysmeansjustonething;therearenomodestokeeptrackof.U+0061isalways'a',evenifalanguagedoesn'thavean'a'init.
Thisappearstobeagreatidea.Oneencodingtorulethemall.Multiplelanguagesperdocument.Nomoremodeswitchingtoswitchbetweenencodingsmid-stream.ButFourbytesforeverysinglecharacter?Thatisreallywasteful,especiallyforlanguageslikeEnglishandSpanish,whichneedlessthanonebyte(256numbers)toexpresseverypossiblecharacter.
Unicode-UTF-32
ThereisaUnicodeencodingthatusesfourbytespercharacter.It'scalledUTF-32,because32bits=4bytes.UTF-32isastraightforwardencoding;ittakeseachUnicodecharacter(a4-bytenumber)andrepresentsthecharacterwiththatsamenumber.Thishassomeadvantages,themostimportantbeingthatwecanfindtheNthcharacterofastringinconstanttime,becausetheNthcharacterstartsatthe4xNthbyte.Italsohasseveraldisadvantages,themostobviousbeingthatittakesfourfreakingbytestostoreeveryfreakingcharacter.
Unicode-UTF-16
EventhoughtherearealotofUnicodecharacters,itturnsoutthatmostpeoplewillneveruseanythingbeyondthefirst65535.Thus,thereisanotherUnicodeencoding,calledUTF-16(because16bits=2bytes).UTF-16encodeseverycharacterfrom0-65535astwobytes,thenusessomedirtyhacksifweactuallyneedtorepresenttherarely-usedUnicodecharactersbeyond65535.Mostobviousadvantage:UTF-16istwiceasspace-efficientasUTF-32,becauseeverycharacterrequiresonlytwobytestostoreinsteadoffourbytes.AndwecanstilleasilyfindtheNthcharacterofastringinconstanttime.
Buttherearealsonon-obviousdisadvantagestobothUTF-32andUTF-16.Differentcomputersystemsstoreindividualbytesindifferentways.ThatmeansthatthecharacterU+4E2DcouldbestoredinUTF-16aseither4E2Dor2D4E,dependingonwhetherthesystemisbig-endianorlittle-endian.(ForUTF-32,thereareevenmorepossiblebyteorderings.)
Tosolvethisproblem,themulti-byteUnicodeencodingsdefineaByteOrderMark,whichisaspecialnon-printablecharacterthatwecanincludeatthebeginningofourdocumenttoindicatewhatorderourbytesarein.ForUTF-16,theByteOrderMarkisU+FEFF.IfwereceiveaUTF-16documentthatstartswiththebytesFFFE,weknowthebyteorderingisoneway;ifitstartswithFEFF,weknowthebyteorderingisreversed.
Still,UTF-16isn'texactlyideal,especiallyifwe'redealingwithalotofASCIIcharacters.Ifwethinkaboutit,evenaChinesewebpageisgoingtocontainalotofASCIIcharacters-alltheelementsandattributessurroundingtheprintableChinesecharacters.BeingabletofindtheNthcharacterinconstanttimeisnice,butwecan'tguaranteethateverycharacterisexactlytwobytes,sowecan'treallyfindtheNthcharacterinconstanttimeunlesswemaintainaseparateindex.
Unicode-UTF-8
UTF-8isavariable-lengthencodingsystemforUnicode.Thatis,differentcharacterstakeupadifferentnumberofbytes.ForASCIIcharacters(A-Z)UTF-8usesjustonebytepercharacter.Infact,itusestheexactsamebytes;thefirst128characters(0-127)inUTF-8areindistinguishablefromASCII.ExtendedLatincharacterslikeñandüenduptakingtwobytes.(ThebytesarenotsimplytheUnicodecodepointliketheywouldbeinUTF-16;thereissomeseriousbit-twiddlinginvolved.)Chinesecharacterslikeçenduptakingthreebytes.Therarely-usedastralplanecharacterstakefourbytes.
Disadvantages:becauseeachcharactercantakeadifferentnumberofbytes,findingtheNthcharacterisanO(N)operation-thatis,thelongerthestring,thelongerittakestofindaspecificcharacter.Also,thereisbit-twiddlinginvolvedtoencodecharactersintobytesanddecodebytesintocharacters.
Advantages:super-efficientencodingofcommonASCIIcharacters.NoworsethanUTF-16forextendedLatincharacters.BetterthanUTF-32forChinesecharacters.Alsotherearenobyte-orderingissues.Adocumentencodedinutf-8usestheexactsamestreamofbytesonanycomputer.
FileObject
Theopen()functionreturnsafileobject,whichhasmethodsandattributesforgettinginformationaboutandmanipulatingastreamofcharacters.
>>>file=open('Alone.txt')
>>>file.mode
'r'
>>>file.name
'Alone.txt'
>>>file.encoding
'cp1252'
Ifwespecifytheencoding:
>>>#-*-coding:utf-8-*-
>>>file=open('Alone.txt',encoding='utf-8')
>>>file.encoding
'utf-8'
>>>str=file.read()
>>>str
'나혼자(Alone)-BySistar\n추억이이리많을까넌대체뭐할까\n아직난이래혹시돌아올까봐\n'
ThefirstlinewasencodingdeclarationwhichneededtomakethePythonawareofKorean.
Thenameattributereflectsthenamewepassedintotheopen()functionwhenweopenedthefile.Theencodingattributereflectstheencodingwepassedintotheopen()function.Ifwedidn'tspecifytheencodingwhenweopenedthefile,thentheencodingattributewillreflectlocale.getpreferredencoding().Themodeattributetellsusinwhichmodethefilewasopened.Wecanpassanoptionalmodeparametertotheopen()function.Wedidn'tspecifyamodewhenweopenedthisfile,soPythondefaultsto'r',whichmeansopenforreadingonly,intextmode.Thefilemodeservesseveralpurposes;differentmodesletuswritetoafile,appendtoafile,oropenafileinbinarymode.
read()
>>>file=open('Alone.txt',encoding='utf-8')
>>>str=file.read()
>>>str
'나혼자(Alone)-BySistar\n추억이이리많을까넌대체뭐할까\n아직난이래혹시돌아올까봐\n'
>>>file.read()
''
Readingthefileagaindoesnotraiseanexception.Pythondoesnotconsiderreadingpastend-of-filetobeanerror;itsimplyreturnsanemptystring.
>>>file.read()
''
Sincewe'restillattheendofthefile,furthercallstothestreamobject'sread()methodsimplyreturnanemptystring.
>>>file.seek(0)
0
Theseek()methodmovestoaspecificbytepositioninafile.
>>>file.read(10)
'나혼자(Alon'
>>>file.seek(0)
0
>>>file.read(15)
'나혼자(Alone)-'
>>>file.read(1)
'B'
>>>file.read(10)
'ySistar\n추'
>>>file.tell()
34
Theread()methodcantakeanoptionalparameter,thenumberofcharacterstoread.Wecanalsoreadonecharacteratatime.Theseek()andtell()methodsalwayscountbytes,butsinceweopenedthisfileastext,theread()methodcountscharacters.KoreancharactersrequiremultiplebytestoencodeinUTF-8.TheEnglishcharactersinthefileonlyrequireonebyteeach,sowemightbemisledintothinkingthattheseek()andread()methodsarecountingthesamething.Butthat'sonlytrueforsomecharacters.
close()
It'simportanttoclosefilesassoonaswe'redonewiththembecauseopenfilesconsumesystemresources,anddependingonthefilemode,otherprogramsmaynotbeabletoaccessthem.
>>>file.close()
>>>file.read()
Traceback(mostrecentcalllast):
File"",line1,in
file.read()
ValueError:I/Ooperationonclosedfile.
>>>file.seek(0)
Traceback(mostrecentcalllast):
File"",line1,in
file.seek(0)
ValueError:I/Ooperationonclosedfile.
>>>file.tell()
Traceback(mostrecentcalllast):
File"",line1,in
file.tell()
ValueError:I/Ooperationonclosedfile.
>>>file.close()
>>>file.closed
True
Wecan'treadfromaclosedfile;thatraisesanIOErrorexception.
Wecan'tseekinaclosedfileeither.
There'snocurrentpositioninaclosedfile,sothetell()methodalsofails.
Callingtheclose()methodonastreamobjectwhosefilehasbeencloseddoesnotraiseanexception.It'sjustano-op.
Closedstreamobjectsdohaveoneusefulattribute:theclosedattributewillconfirmthatthefileisclosed.
"with"statement
Streamobjectshaveanexplicitclose()method,butwhathappensifourcodehasabugandcrashesbeforewecallclose()?Thatfilecouldtheoreticallystayopenforlongerthannecessary.
Probably,wecouldusethetry..finallyblock.Butwehaveacleanersolution,whichisnowthepreferredsolutioninPython3:thewithstatement:
>>>withopen('Alone.txt',encoding='utf-8')asfile:
file.seek(16)
char=file.read(1)
print(char)
16
o
Thecodeabovenevercallsfile.close().Thewithstatementstartsacodeblock,likeanifstatementoraforloop.Insidethiscodeblock,wecanusethevariablefileasthestreamobjectreturnedfromthecalltoopen().Alltheregularstreamobjectmethodsareavailable-seek(),read(),whateverweneed.Whenthewithblockends,Pythoncallsfile.close()automatically.
Notethatnomatterhoworwhenweexitthewithblock,Pythonwillclosethatfileevenifweexititviaanunhandledexception.Inotherwords,evenifourcoderaisesanexceptionandourentireprogramcomestoahalt,thatfilewillgetclosed.Guaranteed.
Actually,thewithstatementcreatesaruntimecontext.Intheseexamples,thestreamobjectactsasacontextmanager.Pythoncreatesthestreamobjectfileandtellsitthatitisenteringaruntimecontext.Whenthewithcodeblockiscompleted,Pythontellsthestreamobjectthatitisexitingtheruntimecontext,andthestreamobjectcallsitsownclose()method.
There'snothingfile-specificaboutthewithstatement;it'sjustagenericframeworkforcreatingruntimecontextsandtellingobjectsthatthey'reenteringandexitingaruntimecontext.Iftheobjectinquestionisastreamobject,thenitclosesthefileautomatically.Butthatbehaviorisdefinedinthestreamobject,notinthewithstatement.Therearelotsofotherwaystousecontextmanagersthathavenothingtodowithfiles.
Readinglinesonebyone
Alineoftextisasequenceofcharactersdelimitedbywhatexactly?Well,it'scomplicated,becausetextfilescanuseseveraldifferentcharacterstomarktheendofaline.Everyoperatingsystemhasitsownconvention.Someuseacarriagereturncharacter(\r),othersusealinefeedcharacter(\n),andsomeusebothcharacters(\r\n)attheendofeveryline.
However,Pythonhandleslineendingsautomaticallybydefault.Pythonwillfigureoutwhichkindoflineendingthetextfileusesandanditwillalltheworkforus.
#line.py
lineCount=0
withopen('Daffodils.txt',encoding='utf-8')asfile:
forlineinfile:
lineCount+=1
print('{:<5}{}'.format(lineCount,line.rstrip()))
Ifwerunit:
C:\TEST>pythonline.py
1Iwanderedlonelyasacloud
2Thatfloatsonhigho'ervalesandhills,
3WhenallatonceIsawacrowd,
4Ahost,ofgoldendaffodils;
Usingthewithpattern,wesafelyopenthefileandletPythoncloseitforus.
Toreadafileonelineatatime,useaforloop.That'sit.Besideshavingexplicitmethodslikeread(),thestreamobjectisalsoaniteratorwhichspitsoutasinglelineeverytimeweaskforavalue.
Usingtheformat()stringmethod,wecanprintoutthelinenumberandthelineitself.Theformatspecifier{:<5}meansprintthisargumentleft-justifiedwithin5spaces.Thea_linevariablecontainsthecompleteline,carriagereturnsandall.Therstrip()stringmethodremovesthetrailingwhitespace,includingthecarriagereturncharacters.
write()
Wecanwritetofilesinmuchthesamewaythatwereadfromthem.First,weopenafileandgetafileobject,thenweusemethodsonthestreamobjecttowritedatatothefile,thenclosethefile.
Themethodwrite()writesastringtothefile.Thereisnoreturnvalue.Duetobuffering,thestringmaynotactuallyshowupinthefileuntiltheflush()orclose()methodiscalled.
Toopenafileforwriting,usetheopen()functionandspecifythewritemode.Therearetwofilemodesforwritingaslistedintheearliertable:
writemodewilloverwritethefilewhenthemode='w'oftheopen()function.
appendmodewilladddatatotheendofthefilewhenthemode='a'oftheopen()function.
Weshouldalwayscloseafileassoonaswe'redonewritingtoit,toreleasethefilehandleandensurethatthedataisactuallywrittentodisk.Aswithreadingdatafromafile,wecancallthestreamobject'sclose()method,orwecanusethewithstatementandletPythonclosethefileforus.
>>>withopen('myfile',mode='w',encoding='utf-8')asfile:
file.write('Copyandpasteisadesignerror.')
>>>withopen('myfile',encoding='utf-8')asfile:
print(file.read())
Copyandpasteisadesignerror.
>>>
>>>withopen('myfile',mode='a',encoding='utf-8')asfile:
file.write('\nTestingshowsthepresence,nottheabsenceofbugs.')
>>>withopen('myfile',encoding='utf-8')asfile:
print(file.read())
Copyandpasteisadesignerror.
Testingshowsthepresence,nottheabsenceofbugs.
Westartedbycreatingthenewfilemyfile,andopeningthefileforwriting.Themode='w'parametermeansopenthefileforwriting.
Wecanadddatatothenewlyopenedfilewiththewrite()methodofthefileobjectreturnedbytheopen()function.Afterthewithblockends,Pythonautomaticallyclosesthefile.
Then,withmode='a'toappendtothefileinsteadofoverwritingit.Appendingwillneverharmtheexistingcontentsofthefile.Boththeoriginallinewewroteandthesecondlineweappendedarenowinthefile.Alsonotethatneithercarriagereturnsnorlinefeedsareincluded.Notethatwewrotealinefeedwiththe'\n'character.
Binaryfiles
Picturefileisnotatextfile.Binaryfilesmaycontainanytypeofdata,encodedinbinaryformforcomputerstorageandprocessingpurposes.
Binaryfilesareusuallythoughtofasbeingasequenceofbytes,whichmeansthebinarydigits(bits)aregroupedineights.Binaryfilestypicallycontainbytesthatareintendedtobeinterpretedassomethingotherthantextcharacters.Compiledcomputerprogramsaretypicalexamples;indeed,compiledapplications(objectfiles)aresometimesreferredto,particularlybyprogrammers,asbinaries.Butbinaryfilescanalsocontainimages,sounds,compressedversionsofotherfiles,etc.-inshort,anytypeoffilecontentwhatsoever.
Somebinaryfilescontainheaders,blocksofmetadatausedbyacomputerprogramtointerpretthedatainthefile.Forexample,aGIFfilecancontainmultipleimages,andheadersareusedtoidentifyanddescribeeachblockofimagedata.Ifabinaryfiledoesnotcontainanyheaders,itmaybecalledaflatbinaryfile.Butthepresenceofheadersarealsocommoninplaintextfiles,likeemailandhtmlfiles.-wiki
>>>my_image=open('python_image.png',mode='rb')
>>>my_image.mode
'rb'
>>>my_image.name
'python_image.png'
>>>my_image.encoding
Traceback(mostrecentcalllast):
File"",line1,in
my_image.encoding
AttributeError:'_io.BufferedReader'objecthasnoattribute'encoding'
Openingafileinbinarymodeissimplebutsubtle.Theonlydifferencefromopeningitintextmodeisthatthemodeparametercontainsa'b'character.Thestreamobjectwegetfromopeningafileinbinarymodehasmanyofthesameattributes,includingmode,whichreflectsthemodeparameterwepassedintotheopen()function.Binaryfileobjectsalsohaveanameattribute,justliketextfileobjects.
However,abinarystreamobjecthasnoencodingattribute.That'sbecausewe'rereadingbytes,notstrings,sothere'snoconversionforPythontodo.
Let'scontinuetodomoreinvestigationonthebinary:
>>>my_image.tell()
0
>>>image_data=my_image.read(5)
>>>image_data
b'\x89PNG\r'
>>>type(image_data)
>>>my_image.tell()
5
>>>my_image.seek(0)
0
>>>image_data=my_image.read()
>>>len(image_data)
14922
Liketextfiles,wecanreadbinaryfilesalittlebitatatime.Asmentionedpreviously,there'sacrucialdifference.We'rereadingbytes,notstrings.Sinceweopenedthefileinbinarymode,theread()methodtakesthenumberofbytestoread,notthenumberofcharacters.
Thatmeansthatthere'sneveranunexpectedmismatchbetweenthenumberwepassedintotheread()methodandthepositionindexwegetoutofthetell()method.Theread()methodreadsbytes,andtheseek()andtell()methodstrackthenumberofbytesread.
read()
sizeparameter
Wecanreadastreamobjectiswitharead()methodthattakesanoptionalsizeparameter.Then,theread()methodreturnsastringofthatsize.Whencalledwithnosizeparameter,theread()methodshouldreadeverythingthereandreturnallthedataasasinglevalue.Whencalledwithasizeparameter,itreadsthatmuchfromtheinputsourceandreturnsthatmuchdata.Whencalledagain,itpicksupwhereitleftoffandreturnsthenextchunkofdata.
>>>importio
>>>my_string='Cisquirky,flawed,andanenormoussuccess.-DennisRitchie(1941-2011)'
>>>my_file=io.StringIO(my_string)
>>>my_file.read()
'Cisquirky,flawed,andanenormoussuccess.-DennisRitchie(1941-2011)'
>>>my_file.read()
''
>>>my_file.seek(0)
0
>>>my_file.read(10)
'Cisquirk'
>>>my_file.tell()
10
>>>my_file.seek(10)
10
>>>my_file.read()
'y,flawed,andanenormoussuccess.-DennisRitchie(1941-2011)'
TheiomoduledefinestheStringIOclassthatwecanusetotreatastringinmemoryasafile.
Tocreateastreamobjectoutofastring,createaninstanceoftheio.StringIO()classandpassitthestringwewanttouseasourfiledata.Nowwehaveastreamobject,andwecandoallsortsofstream-likethingswithit.
Callingtheread()methodreadstheentirefile,whichinthecaseofaStringIOobjectsimplyreturnstheoriginalstring.
Wecanexplicitlyseektothebeginningofthestring,justlikeseekingthrougharealfile,byusingtheseek()methodoftheStringIOobject.
Wecanalsoreadthestringinchunks,bypassingasizeparametertotheread()method.
Readingcompressedfiles
ThePythonstandardlibrarycontainsmodulesthatsupportreadingandwritingcompressedfiles.Thereareanumberofdifferentcompressionschemes.Thetwomostpopularonnon-Windowssystemsaregzipandbzip2.
Thoughitdependsontheintendedapplication.gzipisveryfastandhassmallmemoryfootprint.bzip2can'tcompetewithgzipintermsofspeedormemoryusage.bzip2hasnotablybettercompressionratiothangzip,whichhastobethereasonforthepopularityofbzip2;itisslowerthangzipespeciallyindecompressionandusesmorememory.
Datafromgzipvsbzip2.
Thegzipmoduleletsuscreateastreamobjectforreadingorwritingagzip-compressedfile.Thestreamobjectitgivesussupportstheread()methodifweopeneditforreadingorthewrite()methodifweopeneditforwriting.Thatmeanswecanusethemethodswe'vealreadylearnedforregularfilestodirectlyreadorwriteagzip-compressedfile,withoutcreatingatemporaryfiletostorethedecompresseddata.
>>>importgzip
>>>withgzip.open('myfile.g',mode='wb')ascompressed:
compressed.write('640Koughttobeenoughforanybody(1981).-BillGates(1981)'.encode('utf-8'))
$ls-lmyfile.gz
-rwx------+1AdministratorsNone82Jan322:38myfile.gz
$gunzipmyfile.gz
$catmyfile
640Koughttobeenoughforanybody(1981).-BillGates(1981)
Weshouldalwaysopengzippedfilesinbinarymode.(Notethe'b'characterinthemodeargument.)
Thegzipfileformatincludesafixed-lengthheaderthatcontainssomemetadataaboutthefile,soit'sinefficientforextremelysmallfiles.
Thegunzipcommanddecompressesthefileandstoresthecontentsinanewfilenamedthesameasthecompressedfilebutwithoutthe.gzfileextension.
Thecatcommanddisplaysthecontentsofafile.Thisfilecontainsthestringwewrotedirectlytothecompressedfilemyfile.gzfromwithinthePythonShell.
stdoutandstderr
Picturefromwiki
stdin,stdout,andstderrarepipesthatarebuiltintoeverysystemsuchasLinuxandMacOSX.Whenwecalltheprint()function,thethingwe'reprintingissenttothestdoutpipe.Whenourprogramcrashesandprintsoutatraceback,itgoestothestderrpipe.Bydefault,bothofthesepipesarejustconnectedtotheterminal.Whenourprogramprintssomething,weseetheoutputinourterminalwindow,andwhenaprogramcrashes,weseethetracebackinourterminalwindowtoo.InthegraphicalPythonShell,thestdoutandstderrpipesdefaulttoourIDEWindow.
>>>forninrange(2):
print('JavaistoJavaScriptwhatCaristoCarpet')
JavaistoJavaScriptwhatCaristoCarpet
JavaistoJavaScriptwhatCaristoCarpet
>>>importsys
>>>forninrange(2):
s=sys.stdout.write('Simplicityisprerequisiteforreliability.')
Simplicityisprerequisiteforreliability.Simplicityisprerequisiteforreliability.
>>>forninrange(2):
s=sys.stderr.write('stderr')
stderrstderr
Thestdoutisdefinedinthesysmodule,anditisastreamobject.Callingitswrite()functionwillprintoutwhateverstringwegive,thenreturnthelengthoftheoutput.Infact,thisiswhattheprint()functionreallydoes;itaddsacarriagereturntotheendofthestringwe'reprinting,andcallssys.stdout.write.
sys.stdoutandsys.stderrsendtheiroutputtothesameplace:thePythonideifwe'rein,ortheterminalifwe'rerunningPythonfromthecommandline.Likestandardoutput,standarderrordoesnotaddcarriagereturnsforus.Ifwewantcarriagereturns,we'llneedtowritecarriagereturncharacters.
Notethatstdoutandstderrarewrite-only.Attemptingtocalltheirread()methodwillalwaysraiseanIOError.
>>>importsys
>>>sys.stdout.read()
Traceback(mostrecentcalllast):
File"",line1,in
sys.stdout.read()
AttributeError:read
stdoutredirect
stdoutandstderronlysupportwritingbutthey'renotconstants.They'revariables!Thatmeanswecanassignthemanewvaluetoredirecttheiroutput.
#redirect.py
importsys
classStdoutRedirect:
def__init__(self,newOut):
self.newOut=newOut
def__enter__(self):
self.oldOut=sys.stdout
sys.stdout=self.newOut
def__exit__(self,*args):
sys.stdout=self.oldOut
print('X')
withopen('output',mode='w',encoding='utf-8')asmyFile:
withStdoutRedirect(myFile):
print('Y')
print('Z')
Ifwerunit:
$pythonredirect.py
X
Z
$catoutput
Y
Weactuallyhavetwowithstatements,onenestedwithinthescopeoftheother.Theouterwithstatementopensautf-8-encodedtextfilenamedoutputforwritingandassignsthestreamobjecttoavariablenamedmyFile.
However,
withStdoutRedirect(myFile):
Where'stheasclause?
Thewithstatementdoesn'tactuallyrequireone.Wecanhaveawithstatementthatdoesn'tassignthewithcontexttoavariable.Inthiscase,we'reonlyinterestedinthesideeffectsoftheStdoutRedirectcontext.
Whatarethosesideeffects?
TakealookinsidetheStdoutRedirectclass.Thisclassisacustomcontextmanager.Anyclasscanbeacontextmanagerbydefiningtwospecialmethods:__enter__()and__exit__().
The__init__()methodiscalledimmediatelyafteraninstanceiscreated.Ittakesoneparameter,thestreamobjectthatwewanttouseasstandardoutputforthelifeofthecontext.Thismethodjustsavesthestreamobjectinaninstancevariablesoothermethodscanuseitlater.
The__enter__()methodisaspecialclassmethod.Pythoncallsitwhenenteringacontext(i.e.atthebeginningofthewithstatement).Thismethodsavesthecurrentvalueofsys.stdoutinself.oldOut,thenredirectsstandardoutputbyassigningself.newOuttosys.stdout.
__exit__()methodisanotherspecialclassmethod.Pythoncallsitwhenexitingthecontext(i.e.attheendofthewithstatement).Thismethodrestoresstandardoutputtoitsoriginalvaluebyassigningthesavedself.oldOutvaluetosys.stdout.
Thiswithstatementtakesacomma-separatedlistofcontexts.Thecomma-separatedlistactslikeaseriesofnestedwithblocks.Thefirstcontextlistedistheouterblock;thelastonelistedistheinnerblock.Thefirstcontextopensafile;thesecondcontextredirectssys.stdouttothestreamobjectthatwascreatedinthefirstcontext.Becausethisprint()functionisexecutedwiththecontextcreatedbythewithstatement,itwillnotprinttothescreen;itwillwritetothefileoutput.
Now,thewithcodeblockisover.Pythonhastoldeachcontextmanagertodowhateveritistheydouponexitingacontext.Thecontextmanagersformalast-in-first-outstack.Uponexiting,thesecondcontextchangedsys.stdoutbacktoitsoriginalvalue,thenthefirstcontextclosedthefilenamedoutput.Sincestandardoutputhasbeenrestoredtoitsoriginalvalue,callingtheprint()functionwillonceagainprinttothescreen.
Fileread/write-sample
Thefollowingexampleshowsanotherexampleofreadingandwriting.Itreadstwodatafile(linuxworddictionary,andtop-levelcountrydomainnamessuchas.us,.lyetc.),andfindthecombinationofthetwoforagivenlengthofthefulldomainname.
#Findingacombinationofwordsanddomainname(.ly,.us,etc).
LENGTH=8
d_list=[]
withopen('domain.txt','r')asdf:
fordindf:
d_list.append((d[0:2]).lower())
printd_list[:10]
d_list=['us','ly']
wf=open('words.txt','r')
w_list=wf.read().split()
wf.close()
printlen(w_list)
printw_list[:10]
withopen('domain_out.txt','w')asoutf:
fordind_list:
print'-------',d,'------\n'
outf.write('-------'+d+'------\n')
forwinw_list:
ifw[-2:]==dandlen(w)==LENGTH:
printw[:-2]+'.'+d
outf.write(w[:-2]+'.'+d+'\n')
Sampleoutput:
-------us------
...
enormo.us
exiguo.us
fabulo.us
genero.us
glorio.us
gorgeo.us
...
virtuo.us
vitreo.us
wondro.us
-------ly------
Connol.ly
Kimber.ly
Thessa.ly
abject.ly
abrupt.ly
absent.ly
absurd.ly
active.ly
actual.ly
...
keywordfinally
Thekeywordfinallymakesadifferenceifourcodereturnsearly:
try:
code1()
exceptTypeError:
code2()
returnNone
finally:
other_code()
Withthiscode,thefinallyblockisassuredtorunbeforethemethodreturns.Thecaseswhenthiscouldhappen:
Ifanexceptionisthrowninsidetheexceptblock.
Ifanexceptionisthrowninrun_code1()butit'snotaTypeError.
Othercontrolflowstatementssuchascontinueandbreakstatements.
However,withoutthefinallyblock:
try:
run_code1()
exceptTypeError:
run_code2()
returnNone
other_code()
theother_code()doesn'tgetrunifthere'sanexception.
Ph.D./GoldenGateAve,SanFrancisco/SeoulNationalUniv/CarnegieMellon/UCBerkeley/DevOps/DeepLearning/Visualization
SponsorOpenSourcedevelopmentactivitiesandfreecontentsforeveryone.
Thankyou.
-KHong
Pythontutorial
PythonHome
Introduction
RunningPythonPrograms(os,sys,import)
ModulesandIDLE(Import,Reload,exec)
ObjectTypes-Numbers,Strings,andNone
Strings-EscapeSequence,RawString,andSlicing
Strings-Methods
FormattingStrings-expressionsandmethodcalls
Filesandos.path
Traversingdirectoriesrecursively
SubprocessModule
RegularExpressionswithPython
RegularExpressionsCheatSheet
ObjectTypes-Lists
ObjectTypes-DictionariesandTuples
Functionsdef,*args,**kargs
Functionslambda
Built-inFunctions
map,filter,andreduce
Decorators
ListComprehension
Sets(union/intersection)anditertools-Jaccardcoefficientandshinglingtocheckplagiarism
Hashing(Hashtablesandhashlib)
DictionaryComprehensionwithzip
Theyieldkeyword
GeneratorFunctionsandExpressions
generator.send()method
Iterators
ClassesandInstances(__init__,__call__,etc.)
if__name__=='__main__'
argparse
Exceptions
@staticmethodvsclassmethod
Privateattributesandprivatemethods
bits,bytes,bitstring,andconstBitStream
json.dump(s)andjson.load(s)
PythonObjectSerialization-pickleandjson
PythonObjectSerialization-yamlandjson
Priorityqueueandheapqueuedatastructure
Graphdatastructure
Dijkstra'sshortestpathalgorithm
Prim'sspanningtreealgorithm
Closure
FunctionalprogramminginPython
Remoterunningalocalfileusingssh
SQLite3-A.ConnectingtoDB,create/droptable,andinsertdataintoatable
SQLite3-B.Selecting,updatinganddeletingdata
MongoDBwithPyMongoI-InstallingMongoDB...
PythonHTTPWebServices-urllib,httplib2
WebscrapingwithSeleniumforcheckingdomainavailability
RESTAPI:HttpRequestsforHumanswithFlask
BlogappwithTornado
Multithreading...
PythonNetworkProgrammingI-BasicServer/Client:ABasics
PythonNetworkProgrammingI-BasicServer/Client:BFileTransfer
PythonNetworkProgrammingII-ChatServer/Client
PythonNetworkProgrammingIII-EchoServerusingsocketservernetworkframework
PythonNetworkProgrammingIV-AsynchronousRequestHandling:ThreadingMixInandForkingMixIn
PythonCodingQuestionsI
PythonCodingQuestionsII
PythonCodingQuestionsIII
PythonCodingQuestionsIV
PythonCodingQuestionsV
PythonCodingQuestionsVI
PythonCodingQuestionsVII
PythonCodingQuestionsVIII
ImageprocessingwithPythonimagelibraryPillow
PythonandC++withSIP
PyDevwithEclipse
Matplotlib
RediswithPython
NumPyarraybasicsA
NumPyMatrixandLinearAlgebra
PandaswithNumPyandMatplotlib
CelluarAutomata
Batchgradientdescentalgorithm
LongestCommonSubstringAlgorithm
PythonUnitTest-TDDusingunittest.TestCaseclass
Simpletool-Googlepagerankingbykeywords
GoogleAppHelloWorld
GoogleAppwebapp2andWSGI
UploadingGoogleAppHelloWorld
Python2vsPython3
virtualenvandvirtualenvwrapper
UploadingabigfiletoAWSS3usingbotomodule
ScheduledstoppingandstartinganAWSinstance
ClouderaCDH5-Scheduledstoppingandstartingservices
RemovingCloudFiles-RackspaceAPIwithcurlandsubprocess
Checkingifaprocessisrunning/hangingandstop/runascheduledtaskonWindows
ApacheSpark1.3withPySpark(SparkPythonAPI)Shell
ApacheSpark1.2Streaming
bottle0.12.7-FastandsimpleWSGI-microframeworkforsmallweb-applications...
FlaskappwithApacheWSGIonUbuntu14/CentOS7...
SeleniumWebDriver
Fabric-streamliningtheuseofSSHforapplicationdeployment
AnsibleQuickPreview-SettingupwebserverswithNginx,configureenviroments,anddeployanApp
NeuralNetworkswithbackpropagationforXORusingonehiddenlayer
NLP-NLTK(NaturalLanguageToolkit)...
RabbitMQ(Messagebrokerserver)andCelery(Taskqueue)...
OpenCV3andMatplotlib...
Simpletool-ConcatenatingslidesusingFFmpeg...
iPython-SignalProcessingwithNumPy
iPythonandJupyter-InstallJupyter,iPythonNotebook,drawingwithMatplotlib,andpublishingittoGithub
iPythonandJupyterNotebookwithEmbeddedD3.js
DownloadingYouTubevideosusingyoutube-dlembeddedwithPython
MachineLearning:scikit-learn...
Django1.6/1.8WebFramework...
SponsorOpenSourcedevelopmentactivitiesandfreecontentsforeveryone.
Thankyou.
-KHong
OpenCV3imageandvideoprocessingwithPython
OpenCV3withPython
Image-OpenCVBGR:MatplotlibRGB
Basicimageoperations-pixelaccess
iPython-SignalProcessingwithNumPy
SignalProcessingwithNumPyI-FFTandDFTforsine,squarewaves,unitpulse,andrandomsignal
SignalProcessingwithNumPyII-ImageFourierTransform:FFT&DFT
InverseFourierTransformofanImagewithlowpassfilter:cv2.idft()
ImageHistogram
VideoCaptureandSwitchingcolorspaces-RGB/HSV
AdaptiveThresholding-Otsu'sclustering-basedimagethresholding
EdgeDetection-SobelandLaplacianKernels
CannyEdgeDetection
HoughTransform-Circles
WatershedAlgorithm:Marker-basedSegmentationI
WatershedAlgorithm:Marker-basedSegmentationII
Imagenoisereduction:Non-localMeansdenoisingalgorithm
Imageobjectdetection:FacedetectionusingHaarCascadeClassifiers
Imagesegmentation-ForegroundextractionGrabcutalgorithmbasedongraphcuts
ImageReconstruction-Inpainting(Interpolation)-FastMarchingMethods
Video:Meanshiftobjecttracking
MachineLearning:Clustering-K-MeansclusteringI
MachineLearning:Clustering-K-MeansclusteringII
MachineLearning:Classification-k-nearestneighbors(k-NN)algorithm
MachineLearningwithscikit-learn
scikit-learninstallation
scikit-learn:Featuresandfeatureextraction-irisdataset
scikit-learn:MachineLearningQuickPreview
scikit-learn:DataPreprocessingI-Missing/Categoricaldata
scikit-learn:DataPreprocessingII-Partitioningadataset/Featurescaling/FeatureSelection/Regularization
scikit-learn:DataPreprocessingIII-DimensionalityreductionvisSequentialfeatureselection/Assessingfeatureimportanceviarandomforests
DataCompressionviaDimensionalityReductionI-Principalcomponentanalysis(PCA)
scikit-learn:DataCompressionviaDimensionalityReductionII-LinearDiscriminantAnalysis(LDA)
scikit-learn:DataCompressionviaDimensionalityReductionIII-Nonlinearmappingsviakernelprincipalcomponent(KPCA)analysis
scikit-learn:LogisticRegression,Overfitting®ularization
scikit-learn:SupervisedLearning&UnsupervisedLearning-e.g.UnsupervisedPCAdimensionalityreductionwithirisdataset
scikit-learn:Unsupervised_Learning-KMeansclusteringwithirisdataset
scikit-learn:LinearlySeparableData-LinearModel&(Gaussian)radialbasisfunctionkernel(RBFkernel)
scikit-learn:DecisionTreeLearningI-Entropy,Gini,andInformationGain
scikit-learn:DecisionTreeLearningII-ConstructingtheDecisionTree
scikit-learn:RandomDecisionForestsClassification
scikit-learn:SupportVectorMachines(SVM)
scikit-learn:SupportVectorMachines(SVM)II
FlaskwithEmbeddedMachineLearningI:SerializingwithpickleandDBsetup
FlaskwithEmbeddedMachineLearningII:BasicFlaskApp
FlaskwithEmbeddedMachineLearningIII:EmbeddingClassifier
FlaskwithEmbeddedMachineLearningIV:Deploy
FlaskwithEmbeddedMachineLearningV:Updatingtheclassifier
scikit-learn:SampleofaspamcommentfilterusingSVM-classifyingagoodoneorabadone
Machinelearningalgorithmsandconcepts
Batchgradientdescentalgorithm
SingleLayerNeuralNetwork-PerceptronmodelontheIrisdatasetusingHeavisidestepactivationfunction
Batchgradientdescentversusstochasticgradientdescent
SingleLayerNeuralNetwork-AdaptiveLinearNeuronusinglinear(identity)activationfunctionwithbatchgradientdescentmethod
SingleLayerNeuralNetwork:AdaptiveLinearNeuronusinglinear(identity)activationfunctionwithstochasticgradientdescent(SGD)
LogisticRegression
VC(Vapnik-Chervonenkis)DimensionandShatter
Bias-variancetradeoff
MaximumLikelihoodEstimation(MLE)
NeuralNetworkswithbackpropagationforXORusingonehiddenlayer
minHash
tf-idfweight
NaturalLanguageProcessing(NLP):SentimentAnalysisI(IMDb&bag-of-words)
NaturalLanguageProcessing(NLP):SentimentAnalysisII(tokenization,stemming,andstopwords)
NaturalLanguageProcessing(NLP):SentimentAnalysisIII(training&crossvalidation)
NaturalLanguageProcessing(NLP):SentimentAnalysisIV(out-of-core)
Locality-SensitiveHashing(LSH)usingCosineDistance(CosineSimilarity)
ArtificialNeuralNetworks(ANN)
[Note]SourcesareavailableatGithub-Jupyternotebookfiles
1.Introduction
2.ForwardPropagation
3.GradientDescent
4.BackpropagationofErrors
5.Checkinggradient
6.TrainingviaBFGS
7.Overfitting&Regularization
8.DeepLearningI:ImageRecognition(Imageuploading)
9.DeepLearningII:ImageRecognition(Imageclassification)
10-DeepLearningIII:DeepLearningIII:Theano,TensorFlow,andKeras
Cp1252 encoding Python