python - Read specific columns from a csv file with csv module?

and I'm expecting that this will print out only the specific columns I want for each row except it doesn't, I get the last column only. python · csv · Share. Home Public Questions Tags Users Companies Collectives ExploreCollectives Teams StackOverflowforTeams –Startcollaboratingandsharingorganizationalknowledge. CreateafreeTeam WhyTeams? Teams CreatefreeTeam Collectives™onStackOverflow Findcentralized,trustedcontentandcollaboratearoundthetechnologiesyouusemost. LearnmoreaboutCollectives Teams Q&Aforwork Connectandshareknowledgewithinasinglelocationthatisstructuredandeasytosearch. LearnmoreaboutTeams Readspecificcolumnsfromacsvfilewithcsvmodule? AskQuestion Asked 9years,5monthsago Modified 1monthago Viewed 782ktimes 204 I'mtryingtoparsethroughacsvfileandextractthedatafromonlyspecificcolumns. Examplecsv: ID|Name|Address|City|State|Zip|Phone|OPEID|IPEDS| 10|C...|130W..|Mo..|AL...|3..|334..|01023|10063| I'mtryingtocaptureonlyspecificcolumns,sayID,Name,ZipandPhone. CodeI'velookedathasledmetobelieveIcancallthespecificcolumnbyitscorrespondingnumber,soie:Namewouldcorrespondto2anditeratingthrougheachrowusingrow[2]wouldproducealltheitemsincolumn2.Onlyitdoesn't. Here'swhatI'vedonesofar: importsys,argparse,csv fromsettingsimport* #commandarguments parser=argparse.ArgumentParser(description='csvtopostgres',\ fromfile_prefix_chars="@") parser.add_argument('file',help='csvfiletoimport',action='store') args=parser.parse_args() csv_file=args.file #opencsvfile withopen(csv_file,'rb')ascsvfile: #getnumberofcolumns forlineincsvfile.readlines(): array=line.split(',') first_item=array[0] num_columns=len(array) csvfile.seek(0) reader=csv.reader(csvfile,delimiter='') included_cols=[1,2,6,7] forrowinreader: content=list(row[i]foriinincluded_cols) printcontent andI'mexpectingthatthiswillprintoutonlythespecificcolumnsIwantforeachrowexceptitdoesn't,Igetthelastcolumnonly. pythoncsv Share Improvethisquestion Follow editedDec6,2016at20:16 martineau 115k2525goldbadges160160silverbadges284284bronzebadges askedMay12,2013at2:10 frankVfrankV 5,09377goldbadges3333silverbadges4545bronzebadges 4 1 why'rb'flagtoopen()?shouldn'titbesimpler? – Elazar May12,2013at2:13 7 @Elazar:inPython2(whichtheOPisusing)"rb"isappropriateforpassingtocsv.reader. – DSM May12,2013at2:14 WhydoesyourexampleCSVfileshowthepipecharacterasthedelimiterbutyourexamplecodeuseaspace? – KellyS.French Sep30,2015at14:06 2 @KellyS.FrenchIthoughtitwouldhelpvisualizethedataforthepurposesofthisquestion. – frankV Apr22,2017at15:37 Addacomment  |  14Answers 14 Sortedby: Resettodefault Highestscore(default) Trending(recentvotescountmore) Datemodified(newestfirst) Datecreated(oldestfirst) 219 Theonlywayyouwouldbegettingthelastcolumnfromthiscodeisifyoudon'tincludeyourprintstatementinyourforloop. Thisismostlikelytheendofyourcode: forrowinreader: content=list(row[i]foriinincluded_cols) printcontent Youwantittobethis: forrowinreader: content=list(row[i]foriinincluded_cols) printcontent Nowthatwehavecoveredyourmistake,Iwouldliketotakethistimetointroduceyoutothepandasmodule. Pandasisspectacularfordealingwithcsvfiles,andthefollowingcodewouldbeallyouneedtoreadacsvandsaveanentirecolumnintoavariable: importpandasaspd df=pd.read_csv(csv_file) saved_column=df.column_name#youcanalsousedf['column_name'] soifyouwantedtosavealloftheinfoinyourcolumnNamesintoavariable,thisisallyouneedtodo: names=df.Names It'sagreatmoduleandIsuggestyoulookintoit.Ifforsomereasonyourprintstatementwasinforloopanditwasstillonlyprintingoutthelastcolumn,whichshouldn'thappen,butletmeknowifmyassumptionwaswrong.Yourpostedcodehasalotofindentationerrorssoitwashardtoknowwhatwassupposedtobewhere.Hopethiswashelpful! Share Improvethisanswer Follow answeredMay12,2013at3:06 RyanSaxeRyanSaxe 16.4k2222goldbadges7676silverbadges124124bronzebadges 3 1 Isitpossibletoremovetheindexnumbersfromthequery?@RyanSaxe – MalachiBazar Nov15,2016at20:28 Yes,justiteratethroughitinaforloop. – davegallant Jul7,2017at14:57 "Nowthatwehavecoveredyourmistake,Iwouldliketotakethistimetointroduceyoutothepandasmodule."Ah,yesyou'renotdoingPythonuntilyou'veusedPandas! – als0052 May26at13:36 Addacomment  |  128 importcsv fromcollectionsimportdefaultdict columns=defaultdict(list)#eachvalueineachcolumnisappendedtoalist withopen('file.txt')asf: reader=csv.DictReader(f)#readrowsintoadictionaryformat forrowinreader:#readarowas{column1:value1,column2:value2,...} for(k,v)inrow.items():#goovereachcolumnnameandvalue columns[k].append(v)#appendthevalueintotheappropriatelist #basedoncolumnnamek print(columns['name']) print(columns['phone']) print(columns['street']) Withafilelike name,phone,street Bob,0893,32Silly James,000,400McHilly Smithers,4442,23LoopedSt. Willoutput >>> ['Bob','James','Smithers'] ['0893','000','4442'] ['32Silly','400McHilly','23LoopedSt.'] Oralternativelyifyouwantnumericalindexingforthecolumns: withopen('file.txt')asf: reader=csv.reader(f) next(reader) forrowinreader: for(i,v)inenumerate(row): columns[i].append(v) print(columns[0]) >>> ['Bob','James','Smithers'] Tochangethedeliminatoradddelimiter=""totheappropriateinstantiation,i.ereader=csv.reader(f,delimiter="") Share Improvethisanswer Follow editedJun5at8:37 Subash 81388silverbadges1818bronzebadges answeredMay12,2013at2:34 HennyHHennyH 7,62422goldbadges2828silverbadges3838bronzebadges 0 Addacomment  |  32 Usepandas: importpandasaspd my_csv=pd.read_csv(filename) column=my_csv.column_name #youcanalsousemy_csv['column_name'] Discardunneededcolumnsatparsetime: my_filtered_csv=pd.read_csv(filename,usecols=['col1','col3','col7']) P.S.I'mjustaggregatingwhatother'shavesaidinasimplemanner.Actualanswersaretakenfromhereandhere. Share Improvethisanswer Follow editedFeb17,2019at16:56 answeredMay23,2017at9:05 VasiliNovikovVasiliNovikov 9,00033goldbadges4545silverbadges5959bronzebadges 2 2 IthinkPandasisaperfectlyacceptablesolution.IusePandasoftenandreallylikethelibrary,butthisquestionspecificallyreferencedtheCSVmodule. – frankV May23,2017at17:05 1 @frankVWell,thetitle,thetagsandthefirstparagraphdonotforbidpandasinanyway,AFAIcansee.I'veactuallyjusthopedtoaddasimpleranswertothosealreadymadehere(otheranswersusepandas,too). – VasiliNovikov May23,2017at18:02 Addacomment  |  21 Youcanusenumpy.loadtext(filename).Forexampleifthisisyourdatabase.csv: ID|Name|Address|City|State|Zip|Phone|OPEID|IPEDS| 10|Adam|130W..|Mo..|AL...|3..|334..|01023|10063| 10|Carl|130W..|Mo..|AL...|3..|334..|01023|10063| 10|Adolf|130W..|Mo..|AL...|3..|334..|01023|10063| 10|Den|130W..|Mo..|AL...|3..|334..|01023|10063| AndyouwanttheNamecolumn: importnumpyasnp b=np.loadtxt(r'filepath\name.csv',dtype=str,delimiter='|',skiprows=1,usecols=(1,)) >>>b array(['Adam','Carl','Adolf','Den'], dtype='|S7') Moreeasilyyoucanusegenfromtext: b=np.genfromtxt(r'filepath\name.csv',delimiter='|',names=True,dtype=None) >>>b['Name'] array(['Adam','Carl','Adolf','Den'], dtype='|S7') Share Improvethisanswer Follow editedJan16,2014at16:19 answeredJan10,2014at13:46 GMGM 18.7k1010goldbadges7676silverbadges7979bronzebadges 1 @GIstheremeanttobeanrbeside'filepath\name.csv'? – 114 Dec11,2014at21:22 Addacomment  |  19 Withpandasyoucanuseread_csvwithusecolsparameter: df=pd.read_csv(filename,usecols=['col1','col3','col7']) Example: importpandasaspd importio s=''' total_bill,tip,sex,smoker,day,time,size 16.99,1.01,Female,No,Sun,Dinner,2 10.34,1.66,Male,No,Sun,Dinner,3 21.01,3.5,Male,No,Sun,Dinner,3 ''' df=pd.read_csv(io.StringIO(s),usecols=['total_bill','day','size']) print(df) total_billdaysize 016.99Sun2 110.34Sun3 221.01Sun3 Share Improvethisanswer Follow answeredDec6,2016at20:26 ayhanayhan 66.4k1717goldbadges173173silverbadges191191bronzebadges Addacomment  |  7 Context:Forthistypeofworkyoushouldusetheamazingpythonpetllibrary.Thatwillsaveyoualotofworkandpotentialfrustrationfromdoingthings'manually'withthestandardcsvmodule.AFAIK,theonlypeoplewhostillusethecsvmodulearethosewhohavenotyetdiscoveredbettertoolsforworkingwithtabulardata(pandas,petl,etc.),whichisfine,butifyouplantoworkwithalotofdatainyourcareerfromvariousstrangesources,learningsomethinglikepetlisoneofthebestinvestmentsyoucanmake.Togetstartedshouldonlytake30minutesafteryou'vedonepipinstallpetl.Thedocumentationisexcellent. Answer:Let'ssayyouhavethefirsttableinacsvfile(youcanalsoloaddirectlyfromthedatabaseusingpetl).Thenyouwouldsimplyloaditanddothefollowing. frompetlimportfromcsv,look,cut,tocsv #Loadthetable table1=fromcsv('table1.csv') #Alterthecolums table2=cut(table1,'Song_Name','Artist_ID') #haveaquicklooktomakesurethingsareok.Printsanicelyformattedtabletoyourconsole printlook(table2) #Savetonewfile tocsv(table2,'new.csv') Share Improvethisanswer Follow editedJul20,2016at0:37 TshilidziMudau 6,83366goldbadges3535silverbadges4848bronzebadges answeredMay29,2015at12:19 PeteBeatPeteBeat 27144silverbadges1111bronzebadges Addacomment  |  5 Ithinkthereisaneasierway importpandasaspd dataset=pd.read_csv('table1.csv') ftCol=dataset.iloc[:,0].values Soinhereiloc[:,0],:meansallvalues,0meansthepositionofthecolumn. intheexamplebelowIDwillbeselected ID|Name|Address|City|State|Zip|Phone|OPEID|IPEDS| 10|C...|130W..|Mo..|AL...|3..|334..|01023|10063| Share Improvethisanswer Follow answeredFeb13,2020at11:38 NuriddinKudratovNuriddinKudratov 42155silverbadges1111bronzebadges 0 Addacomment  |  3 importpandasaspd csv_file=pd.read_csv("file.csv") column_val_list=csv_file.column_name._ndarray_values Share Improvethisanswer Follow answeredMay30,2019at16:58 HariKHariK 22133silverbadges88bronzebadges 1 You'llhavetopipinstallpandasfirst – BorisVerkhovskiy Apr26,2020at14:51 Addacomment  |  1 Thankstothewayyoucanindexandsubsetapandasdataframe,averyeasywaytoextractasinglecolumnfromacsvfileintoavariableis: myVar=pd.read_csv('YourPath',sep=",")['ColumnName'] Afewthingstoconsider: ThesnippetabovewillproduceapandasSeriesandnotdataframe. Thesuggestionfromayhanwithusecolswillalsobefasterifspeedisanissue. Testingthetwodifferentapproachesusing%timeitona2122KBsizedcsvfileyields22.8msfortheusecolsapproachand53msformysuggestedapproach. Anddon'tforgetimportpandasaspd Share Improvethisanswer Follow answeredDec10,2018at8:33 vestlandvestland 47.4k3131goldbadges158158silverbadges259259bronzebadges Addacomment  |  1 Ifyouneedtoprocessthecolumnsseparately,Iliketodestructurethecolumnswiththezip(*iterable)pattern(effectively"unzip").Soforyourexample: ids,names,zips,phones=zip(*( (row[1],row[2],row[6],row[7]) forrowinreader )) Share Improvethisanswer Follow editedJan17,2019at19:43 answeredJan15,2019at18:59 RobertJensenRobertJensen 1122bronzebadges Addacomment  |  1 importpandasaspd dataset=pd.read_csv('Train.csv') X=dataset.iloc[:,1:-1].values y=dataset.iloc[:,-1].values Xisaabunchofcolumns,useitifyouwanttoreadmorethatonecolumn yissinglecolumn,useittoreadonecolumn [:,1:-1]are[row_index:to_row_index,column_index:to_column_index] Share Improvethisanswer Follow editedNov21,2021at23:41 Tonechas 13k1515goldbadges4242silverbadges7676bronzebadges answeredNov20,2021at11:21 LalanKumarLalanKumar 1111bronzebadge Addacomment  |  0 SAMPLE.CSV a,1,+ b,2,- c,3,* d,4,/ column_names=["Letter","Number","Symbol"] df=pd.read_csv("sample.csv",names=column_names) print(df) OUTPUT LetterNumberSymbol 0a1+ 1b2- 2c3* 3d4/ letters=df.Letter.to_list() print(letters) OUTPUT ['a','b','c','d'] Share Improvethisanswer Follow answeredOct22,2020at15:10 FredFred 16311silverbadge99bronzebadges Addacomment  |  0 importcsv withopen('input.csv',encoding='utf-8-sig')ascsv_file: #thebelowstatementwillskipthefirstrow next(csv_file) reader=csv.DictReader(csv_file) Time_col={'Time':[]} #print(Time_col) forrecordinreader: Time_col['Time'].append(record['Time']) print(Time_col) Share Improvethisanswer Follow editedAug23at10:16 Chris 118k8888goldbadges261261silverbadges242242bronzebadges answeredAug19at5:00 shivalingeshshivalingesh 1 1 WelcometoStackOverflow.Codeisalotmorehelpfulwhenitisaccompaniedbyanexplanation.StackOverflowisaboutlearning,notprovidingsnippetstoblindlycopyandpaste.Pleaseedityouranswerandexplainhowitanswersthespecificquestionbeingasked.SeeHowtoAnswer. – Chris Aug23at10:15 Addacomment  |  -2 Tofetchcolumnname,insteadofusingreadlines()betterusereadline()toavoidloop&readingthecompletefile&storingitinthearray. withopen(csv_file,'rb')ascsvfile: #getnumberofcolumns line=csvfile.readline() first_item=line.split(',') Share Improvethisanswer Follow answeredMay15,2017at13:52 SurenSuren 3777bronzebadges Addacomment  |  YourAnswer ThanksforcontributingananswertoStackOverflow!Pleasebesuretoanswerthequestion.Providedetailsandshareyourresearch!Butavoid…Askingforhelp,clarification,orrespondingtootheranswers.Makingstatementsbasedonopinion;backthemupwithreferencesorpersonalexperience.Tolearnmore,seeourtipsonwritinggreatanswers. Draftsaved Draftdiscarded Signuporlogin SignupusingGoogle SignupusingFacebook SignupusingEmailandPassword Submit Postasaguest Name Email Required,butnevershown PostYourAnswer Discard Byclicking“PostYourAnswer”,youagreetoourtermsofservice,privacypolicyandcookiepolicy Nottheansweryou'relookingfor?Browseotherquestionstaggedpythoncsvoraskyourownquestion. TheOverflowBlog HowtoearnamillionreputationonStackOverflow:beofservicetoothers Therightwaytojobhop(Ep.495) FeaturedonMeta BookmarkshaveevolvedintoSaves Inboximprovements:markingnotificationsasread/unread,andafiltered... Revieweroverboard!Orarequesttoimprovetheonboardingguidancefornew... CollectivesUpdate:RecognizedMembers,Articles,andGitLab Shouldweburninatethe[script]tag? Visitchat Linked 2 TypeError:'_csv.reader'objectisnotsubscriptableanddayspassed -1 python:csvfileswontputdataintoalist 0 HowtocreateafilterinputfunctionwithPython? 0 HowcanIfilterkeywordsfromacsvfileusingPython 0 ValueError,howtotakecolumnofstringsfromanotherdocumentintomyprogram 0 Extract2listsoutofcsvfileinpython 0 Python;extractrowsfromcsvifparticularcolumnisfilled 0 Howtoreaddatafromaspecificcolumninpython? 0 HowtosavecertaincolumnsfromcsvfileasvariablesinPython? -2 splitingtextfilesusingpython Seemorelinkedquestions Related 1697 HowdoIreadfromstdin? 1073 SavePL/pgSQLoutputfromPostgreSQLtoaCSVfile 2025 Howtoreadafileline-by-lineintoalist? 647 CSVfilewrittenwithPythonhasblanklinesbetweeneachrow 1765 Installingspecificpackageversionwithpip 734 DumpaNumPyarrayintoacsvfile 1271 Howtoreadatextfileintoastringvariableandstripnewlines? 999 WritingapandasDataFrametoCSVfile 157 Howtoextractonecolumnofacsvfile HotNetworkQuestions propernotationforderivativesanddifferential Changelinkcolorbasedinbackgroundcolor? StandardCoverflow-safearithmeticfunctions MakeaCourtTranscriber Howtotellifmybikehasanaluminumframe meaningof'illesas'inMagnaCarta ArethereanyspellsotherthanWishthatcanlocateanobjectthroughleadshielding? DoestheDemocraticPartyofficiallysupportrepealingtheSecondAmendment? PreferenceofBJTtoMOSFET Theunusualphrasing"verb+the+comparativeadjective"intheLordoftheRingsnovels Whatdoyoucallastatementthatisgivenasanexplanationwhysomeonehaswonanaward? Doublelinemathsentence Findanddeletepartiallyduplicatelines Whydoes«facture»mean"bill,invoice"? Wordsforrestaurant WillIgetdeniedentryafterIremovedavisasticker?Ismypassportdamaged? Howtoformalizeagamewhereeachplayerisaprogramhavingaccesstoopponent'scode? HowdoGPSreceiverscommunicatewithsatellites? HowtogetridofUbuntuProadvertisementwhenupdatingapt? Botchingcrosswindlandings Alternativeversionsofbreathing? Vivadoconstraintswizardsuggestsalotofnonsensegeneratedclocks HowtofindthebordercrossingtimeofatraininEurope?(Czechbureaucracyedition) Whyare"eat"and"drink"differentwordsinlanguages? morehotquestions Questionfeed SubscribetoRSS Questionfeed TosubscribetothisRSSfeed,copyandpastethisURLintoyourRSSreader. lang-py Yourprivacy Byclicking“Acceptallcookies”,youagreeStackExchangecanstorecookiesonyourdeviceanddiscloseinformationinaccordancewithourCookiePolicy. Acceptallcookies Customizesettings