Hi All

I have two tab-delimited files. I want to compare first column of testfile1 to first column of testfile2 to find if items in file2 are in file1 and write it to the new file.

I have a following code but not working!! :(.

f1 = open('testfile1.txt')
#f2 = open('testfile2.txt')

for line in f1:
    a = line.split()
    list1 = a[0].split()
    print list1
print "now printing list 2"
    f2 = open('testfile2.txt')
    for line in f2:
        b = line.split()
        list2 = b[0].split()
        print list2
    
    for i,e1 in enumerate(list1):
        for e2 in (list2):
            if e2 in e1:
                print ("line %d : %s" % (i,e1))
            else:
                print"No matching entries"
             
f1.close()
f2.close()

Any helps?

Attachments
IBP-1	1
NF-kappaB1-p50:c-Rel	2
HDAC1:Sp1	3
(T3R-beta1)2	4
RFX1:MIBP1	5
PPUR	6
MyoD:E12	7
NFI/CTF	8
ATF-2{pS69}{pS71}	9
ZFX	10
SIF	11
Fra-1:c-Jun	12
NR1B1:RXR-alpha	13
NF-1/L	14
(c-Jun)2	15
DIF-1	16
NeuroD1:NGN3:E2A	17
c-Jun{p}	18
HSF2A	19
LHX2	20
LHX4	21
GKLFisoform1	22
(NR1B1)2	23
E2F:DP:p107	24
RPF1	25
T3R-beta1	26
Sp3	27
NCOR:c-Ets-2	28
Pbx1:HOXB1	29
PTS1R	30
NFIC2	31
IFNEX	32
FXR-isoform3:FXR-isoform3	33
NF-kappaB	34
HNF6	35
AAF1	36
GKLF	37
Ikaros-1	38
E47:Atoh1	39
MBP-1	40
GR	41
RXR-alpha:v-ErbA	42
GP	43
DAX1A	44
Nrf2:MafG	45
XPA	46
BTEB3:CBP	47
ZBRK1	48
CD28RC	49
Prop-1	50
JunD:Fra-2	51
TWIST:E12	52
PXR-isoform1A	53
ZBRK1:brca1	54
CPBP	55
Mzf2	56
BTEB4	57
C/EBPbeta(p35)	58
STAT1alpha:STAT1alpha	59
BTEB2	60
BTEB3	61
ZNF174	62
gammaCAC2	63
gammaCAC1	64
MAZ	65
AP-2rep	66
HNF-1A	67
LEF-1	68
ERR2	69
Barx2	70
FOXN-2	71
ACE2	72
CP2	73
E2F-3a	74
AR	75
GAAP-1	76
Meis-2b	77
Meis-2c	78
RXR-alpha	79
Meis-2a	80
43K	81
cebpe	82
p300:NF-AT2	83
KSR-1	84
AP-2delta	85
CREB1{p}	86
JunD:ATFa	87
HEN1:Lmo4	88
hsa-miR-302a	89
Rel	90
Mad1	91
ATF-1	92
hsf	93
H4TF-1	94
ATF-2	95
Lmx1	96
ATF-4	97
PREP1	98
jdp2:jdp2	99
HNF-4beta	100
p73alpha	101
PEBP2alphaB1	102
brca1:c-Myc	103
GR-alpha	104
MEFV	105
70-75K	106
c-Myb-isoform1	107
PITX2C:CSX	108
ZIC2	109
GliH1	110
NF1	111
arnt:HIF-1alpha	112
hsa-miR-200c	113
SoxLZ	114
dlx3	115
47-kDa	116
hsa-miR-328	117
Pur	118
C/EBPbeta:C/EBPbeta(p20)	119
LUN-isoform1	120
hsa-miR-132	121
FOXJ2	122
NF-E4	123
hsa-miR-125b	124
hsa-miR-136	125
hsa-miR-125a	126
(C/EBPbeta(p20))2	127
PITX2B:CSX	128
HOXB7	129
AhR	130
Ku80	131
foxm1	132
(c-jun)2	133
NF-1C	134
NF-AT1C	135
NF-1A	136
Hp65	137
ZEB1	138
ING1	139
(RXR-alpha)2	140
STAT3:Fra-1	141
CBF	142
TFAP4	143
Tbx3	144
(CREM)2	145
STAT5{pY}	146
(STAT1)2	147
v-ErbA	148
hsa-miR-143	149
Sp1	150
RAR-gamma	151
HNF-3beta:HNF3A	152
hsa-miR-223	153
Sp2	154
hsa-miR-221	155
hsa-miR-302d	156
GLI	157
Ikaros-2	158
TRERF1	159
RXR-alpha:PPARgamma2	160
ATF-1:CREB1	161
RXR-alpha:VDR	162
Bach1	163
JunB	164
(GATA-2)2	165
JunD	166
CDP-isoform1	167
ZBP99	168
CDP-isoform4	169
NF-IL-2A	170
NF-IL-2B	171
TLS	172
RXR-alpha:PPARalpha	173
SPI1	174
PITX2A:CSX	175
E2A:NeuroD1	176
(COUP-TF2)2	177
GR-beta	178
RF-Y	179
hsa-miR-21	180
SRC-1:c-Ets-1	181
deltaCREB	182
Staf	183
E2F	184
T3R-alpha1:RXR-beta	185
CAR	186
PTTG1	187
NF-jun	188
Nrf2	189
COUP-TF2	190
Prep1	191
Sox18	192
ER-beta	193
AML2	194
AML3	195
Pbx2:Prep1	196
AML1	197
E2F-6	198
E2F-5	199
E2F-4	200
E2F-3	201
E2F-2	202
E2F-1	203
NR1B1:RXR-beta	204
CAR2:RXR-alpha	205
HOXA10-PL1	206
SRF:SRF	207
RAR-gamma:RXR-gamma	208
80-90K	209
PXR:RXR	210
GABP-alpha	211
(T3R-beta)2	212
alpha-CBF	213
Gfi1	214
hsa-miR-27a	215
c-Rel:NF-kappaB1-p50	216
Six-1	217
(TSN)2	218
hsa-miR-27b	219
p63	220
Six-5	221
NF-YA:C/EBPalpha	222
ctcf:RXR-alpha:T3R-alpha1	223
CIZ8	224
T-bet	225
Sox9:Sox9	226
p190RhoGAP	227
REST	228
RBMS1	229
(NFkappaB1-p50)2	230
Rad9a	231
ALP	232
ER71	233
ELOF-1	234
MEF-2B1	235
DDB1:DDB-2:hnRNPA1-A:Sp1	236
(ATF-2)2	237
TIEG-1	238
FOXO3a:AML1b	239
COUP-TF1	240
p107	241
CAP	242
p100	243
NF-Xc	244
MEF2A-isoform1	245
MITF	246
ERF	247
COE1-xbb1	248
ATF/CREB	249
TFII-I	250
IRF-3:IRF-7:CBP:p300	251
AML1c	252
IRF-8:SPI1	253
TEF-5	254
ADA-NF1	255
GKLFa	256
hoxa9	257
UBTF	258
usf1:USF2	259
EWS:ATF-1	260
Meis-1	261
hnRNPK	262
KLF3	263
hsa-miR-196b	264
STAT1alpha:STAT3	265
IRF-3:IRF-7	266
RXR:T3R-beta	267
FOR2	268
FOR1	269
NOC1:C/EBPbeta	270
CREMalpha	271
SPI1:IRF-4	272
NRF-1	273
HNF-1alpha-A	274
hsa-miR-130b	275
HNF-1alpha-C	276
HNF-1alpha-B	277
Tel-2a	278
Tel-2b	279
Tel-2c	280
sp4	281
RFX1:rfx2	282
HOXD1	283
GABP-alpha:GABP-beta2	284
Nkx2-8	285
beta-adaptin2	286
(GABP-alpha)2:(GABP-beta2)2	287
Ets	288
maf	289
Egr-1:Tax	290
RXR-alpha:PPARdelta	291
hsa-miR-130a	292
NF-X2	293
NF-X3	294
hsa-miR-24	295
PITX2A	296
ER-alpha:Sp1	297
Erg-1	298
usf1:usf1	299
hsa-miR-22	300
HSF1-L	301
STAT3-isoform1	302
(T3R-alpha)2	303
NOR1-isoform1	304
PITX2C	305
CAR:RXR-alpha	306
Myogenin	307
PSE1	308
YB-1:Ref-1	309
pax6	310
BEF-1	311
(GABP-alpha)2	312
HSF1-S	313
(RelA-p65)2	314
CIZ	315
Smad3:smad4	316
AR-isoform1	317
ZNFN1A1	318
Fra-2	319
Fra-1	320
FXR:RXR	321
SF-1	322
hsa-miR-218	323
MBD1	324
NF-BA1	325
AP1	326
IRF-7A	327
hsa-miR-657	328
Ikaros	329
p50:p50	330
PEBP2alpha	331
Pax-4c	332
Rb	333
Pax-4a	334
DB1	335
Pit-1A	336
hsa-miR-30b	337
hsa-miR-30c	338
hsa-miR-30d	339
RXR-alpha:T3R-beta1	340
AIC3	341
AIC2	342
(HNF-1alpha)2	343
ER-alpha:HDAC1:Sp1	344
hsa-miR-126	345
AIC5	346
AIC4	347
MOK-2	348
ICER-IIgamma	349
Octa-factor	350
AP-2alpha	351
BMAL1	352
E2F:p130	353
neurofibromin1	354
BMAL2	355
Evi-1	356
TBX5-L	357
pRb	358
CIZ6-1	359
FXR-isoform3	360
ER-alpha	361
Sox5	362
kappaY	363
NFdeltaE4C	364
(GABP-alpha)2:(GABP-beta1)2	365
FKHL5	366
c-Myc:Hsp90	367
WT1	368
RBPJK	369
R1	370
R2	371
CDP:E2F-1	372
GLI2alpha	373
Mad1:Max	374
PNR:NR1B1	375
Elk1-isoform1	376
CP2-isoform1	377
HNF-1beta	378
MAZi	379
ENKTF-1	380
C/EBPbeta(p20)	381
Erm	382
Pbx2	383
CBTF	384
DeltaN	385
Pbx1	386
Atox1	387
MTF-1	388
TCF-1A	389
RFX5	390
NF-kappaB2-p52	391
TCF-1B	392
WT1-del2	393
RFX1	394
TCF-1G	395
RFX3	396
p63gamma	397
irf1:C/EBPbeta	398
RXR-alpha:T3R-alpha	399
DND1	400
arnt:HIF2A	401
hsa-miR-29b	402
HOXA10	403
PLAGL2	404
DBP	405
FXRbeta:RXR-alpha	406
LXR-alpha:RXR-alpha	407
c-Ets-1:GABP-alpha	408
rfx4-isoform4	409
hsa-miR-124a	410
FOXP3	411
RAR-gamma:RXR-beta	412
HNF-4alpha	413
Cdx-1	414
Egr-1:Sp1	415
Cdx-2	416
Tbx3-isoform1	417
Nrf2:JunD	418
hsa-miR-34c	419
STAT3{pS727}	420
AT-BP2	421
SIAHBP1	422
HNF-4	423
HNF-3	424
HNF-1	425
GATA-6short:GATA-6long	426
NGN3:E2A	427
BCL-6	428
HNF-1beta-A	429
RXR-alpha:NR1B1	430
hsa-miR-105	431
p53:CBP	432
Il-6	433
E2F:DP	434
Il-1	435
GABP-alpha:GABP-beta1	436
smad4	437
HoxA5	438
(T3R-alpha1)2	439
PHO2:SWI5	440
HMG2	441
HMG	442
epsilonF1	443
c-Fos:Fra-2:c-Jun	444
Tax:CREB1	445
STAT1beta	446
hsa-let-7b	447
hoxd9	448
pax6{p}	449
p53beta	450
G	451
Sp1:Egr	452
p53-isoform1	453
GTF2IRD1	454
hsa-miR-29a	455
SF1-	456
Miz-1	457
CSX	458
CP1	459
Sp3:c-Ets-1	460
FXR	461
PMX2B	462
nucleolin	463
MyoD	464
ipf1	465
(HNF-4alpha)2	466
CBAF	467
PEBP2alphaA1	468
maf:Nfe2l1	469
Sox10	470
TBP	471
(STAT5A)4	472
Sox13	473
MIBP1	474
PEA3	475
STAT1alpha	476
MEF2C	477
FA Factor Name	478
Oct-2.2	479
CREM	480
Spi-B	481
SPI1:IRF-8	482
CREB	483
c-Myc:Max	484
NF-U1	485
RXR-alpha:NR1B2	486
ATF	487
FIGLA:E2A	488
alpha-enolase	489
BRG1	490
ATF-a:JunD	491
Gscl	492
hsa-miR-127	493
NFdeltaE4A	494
AP-1	495
AP-2	496
NFdeltaE4B	497
AP-4	498
hsa-miR-19b	499
PITX3	500
CHOP-10	501
STAT4:STAT3	502
maf-isoform1	503
hsa-miR-34a	504
C/EBPbeta(LAP)	505
hsa-miR-19a	506
IA-1	507
mammary	508
RORalpha-4	509
MEF-2	510
NFAT5	511
ASP	512
Ref-1	513
Meis-2d	514
GCNF	515
TGIF-isoform2	516
ER81-isoform1	517
hsa-miR-20b	518
IRF-7H	519
Notch1	520
Hey2:HES-1	521
USF2	522
HOXA10:Pbx1a	523
Gzf1	524
Nkx3-1	525
LXR-alpha:PPARalpha	526
JunB:JunB	527
Nfe2l1-isoform2	528
SREBP-2	529
PPARalpha	530
SREBP-1	531
brca1	532
Zbtb5	533
EKLF	534
GAGATA	535
hsa-miR-186	536
hsa-miR-25	537
BBF	538
PRDI-BFc	539
hsa-miR-133a	540
Pbx1:HOXA10	541
hsa-miR-133b	542
arnt2:NXF	543
IRF-9	544
IRF-8	545
p58	546
H1TF1	547
H1TF2	548
FKLF	549
IRF-3	550
IRF-2	551
p50	552
XW	553
IRF-5	554
p53	555
ZEB	556
HELIOS-B	557
Fra-2:c-Jun	558
RXR-alpha:FXR-isoform4	559
ITF	560
KLF8-isoform1	561
hsa-miR-373	562
hsa-miR-23a	563
hsa-miR-23b	564
TWIST	565
HOXA5	566
HDAC1:Sp3:Sp1:p300	567
Smad2:Smad3:smad4	568
Smad	569
AID2	570
hsa-miR-513	571
(TR2-11)2	572
JunD:c-Fos	573
HDBP2	574
HDBP1	575
AAF	576
NF-4FC	577
NF-4FB	578
NF-4FA	579
NCOR:c-Ets-1	580
RPA1	581
COE	582
E12	583
T3R-beta	584
SIAHBP1:FBP	585
gammaCAAT	586
MITF-M1	587
dlx5	588
c-Jun{pS63}{pS73}	589
H4TF-2	590
PITX2	591
NF-AT1	592
NF-AT2	593
NF-AT3	594
NF-AT4	595
HSTF	596
c-Rel:RelA-p65	597
Gli2	598
LEF1-isoform1:beta-catenin	599
OCA-B	600
COUP-TFI	601
Dec-01	602
ATF-2:c-Jun	603
Pax-2.2	604
Pax-2.1	605
mef2d-isoform1	606
SATB1	607
Pbx1a	608
Pbx1b	609
Sp1:PEA3	610
AP-2alphaA	611
PPARgamma:RXR-alpha	612
LRH1-isoform1	613
M-Twist	614
IPCS-BF	615
TBX5-L:CSX	616
hsa-miR-1	617
IRF-4:SPI1	618
Sp1:Tax	619
hsa-miR-7	620
NDPKB	621
hsa-let-7i	622
Oct-02	623
M	624
LRH-1	625
LRH-1-xbb1	626
lmx1b	627
TORC3	628
TORC2	629
TORC1	630
HIF-1alpha	631
Hsp90:c-Myc	632
STAT5A:STAT5B	633
BMAL2-xbb1	634
Max-isoform2	635
IL-10E1	636
TSEB	637
HNF-4alpha1	638
HNF-4alpha2	639
HNF-4alpha3	640
HNF-4alpha4	641
FOXK2a	642
Sp1:NF-YA	643
HNF-1alpha	644
53BP1	645
BP2	646
(TEF-1)2	647
(VDR)2	648
POU2F2	649
POU2F3	650
T3R-alpha:RXR-alpha	651
POU2F1	652
HSF1	653
HDAC1:p130	654
HSF2	655
USF2:usf1	656
DeltaNp73beta	657
RelA-p65:NF-kappaB1	658
egr3	659
hsa-miR-17-5p	660
HiNF-P	661
HNF-3beta	662
HiNF-M	663
litaf-isoform2	664
CDP2	665
RXR-alpha:LXR-beta	666
TCF-4:beta-catenin	667
RelA-p65:c-Rel:p50	668
p55	669
HiNF-D	670
HiNF-E	671
NFAT2-alpha	672
(Pit-1)2	673
HiNF-A	674
HiNF-B	675
HiNF-C	676
ER-alpha:ATF-2	677
HSF	678
T3R-alpha	679
Nrf2:MafK	680
IRF-7	681
RXR:VDR	682
RXR-alpha:NR1B1:NF-AT4:Oct-1	683
JunB:Fra-1	684
JunB:Fra-2	685
SNA	686
HIF2A:arnt	687
POU6F1	688
TREF1	689
TREF2	690
AP3	691
STAT3:STAT3	692
IRF-4	693
REVERB-alpha	694
CBF(2)	695
SRF:Elk-1	696
hsa-miR-520c	697
PPARgamma2:RXR-alpha	698
HOXD8	699
(RXR-beta)2	700
(POU3F2)2	701
Elk-1{p}	702
nerf-isoform2	703
ZIC1	704
RXR-gamma:PPARdelta	705
POU2F1a	706
CREB1	707
mef2a-isoform1	708
PXR	709
HAF-1	710
Sox9	711
Sp1:sp4	712
sin3a	713
hsa-miR-138	714
YB-1	715
Sox2	716
ATBF1-A	717
ATBF1-B	718
nerf-isoform1	719
Sox4	720
MafG	721
HNF-3gamma	722
CAC-binding	723
BTEB1	724
hsa-miR-150	725
ZF2	726
ZF1	727
hsa-miR-497	728
Otx1	729
hsa-miR-206	730
ESE-1	731
Otx2	732
hsa-miR-195	733
TTF-1-isoform1	734
PPAR:RXR	735
Elf-1	736
PR-alpha
ADAR	-1.76258
ZFR	-2.18994
GRLF1	-2.72375
TCF-4	-2.00779
IRF9	-2.72216
HOXB7	-3.07246
SOX11	-2.09344
HOXB2	-3.39292
MEOX1	-3.79881
SIX1	-3.84062
NFATC4	-2.9798
CBFB	-3.25258
HOXD13	-2.86838
POU4F3	-2.39085
NCOA1	-2.79109
MEF2C	-2.65685
NF-kappaB2	1.83054
SHOX2	-2.50137
NCOA1	-2.91833
TADA2L	2.07122
N-Myc	-2.35899
Tbx5	-2.42882
TCF-4	-2.46905
ZBTB7A	-3.72627
MLLT1	-2.46272
BARX1	-3.27179
IRX4	-2.38972
PLXNA1	-2.30475
ZNF335	-2.79064
TCF-4	-2.19378
TSHZ1	-3.26795
TSHZ1	-3.43609
FMNL2	-2.64442
SMAD9	-2.8381
ZNF738	-2.5138
HES2	-2.35944
MLL3	1.90896
ZNF688	-2.73821
ZNF420	2.09812
FOXL1	1.90692

Try this code, which uses list comprehensions

from pprint import pprint

def records(filename):
    """generates pairs (word, line) from the file, where word is the first column"""
    return ((line[:line.find('\t')], line) for line in open(filename))

L1 = list(records('testfile1.txt'))
D1 = dict(L1)

assert(len(L1) == len(D1)) # check that keys are unique in the first file.

pprint(D1)

result = [(word, line) for (word, line) in records('testfile2.txt') if word in D1]

pprint(result)

Edited 6 Years Ago by Gribouillis: n/a

Try this code, which uses list comprehensions

from pprint import pprint

def records(filename):
    """generates pairs (word, line) from the file, where word is the first column"""
    return ((line[:line.find('\t')], line) for line in open(filename))

L1 = list(records('testfile1.txt'))
D1 = dict(L1)

assert(len(L1) == len(D1)) # check that keys are unique in the first file.

pprint(D1)

result = [(word, line) for (word, line) in records('testfile2.txt') if word in D1]

pprint(result)

Thanks a lot for the code.! Working very well.

Is there any other.. simple way to do it. as a beginner it seems bit complicated to rewrite the code on own.

Helps greatly appreciated!

Thanks a lot for the code.! Working very well.

Is there any other.. simple way to do it. as a beginner it seems bit complicated to rewrite the code on own.

Helps greatly appreciated!

Yes, you could write it this way

def records(filename):
    for line in open(filename):
        index = line.find('\t')
        word = line[:index]
        yield (word, line)

D1 = dict(records('testfile1.txt'))

for word, line in records('testfile2.txt'):
    if word in D1:
        # etc... do something

You should learn about the yield statement if you don't know it yet. It is very powerful !

Edited 6 Years Ago by Gribouillis: n/a

You can split records from \t and put first records to set and then same from other file and for each record which is in second file which has same key field put in result. Gribouillis did nicely that he checked for uniqueness of the keys as text file is not database and does not enforce uniqueness.

It is good practise to make generators/list comprehensions as they are pythonic and efficient way of doing things. If you do not like them, you can change them to normal loops easy enough.

result = (line.split()
          for  key in set(line.split('\t',1)[0] for line in open('testfile1.txt'))
          for  line in open('testfile2.txt')
          if line.startswith(key+'\t')
          )
for same in sorted(result):
    print(same)

You can split records from \t and put first records to set and then same from other file and for each record which is in second file which has same key field put in result. Gribouillis did nicely that he checked for uniqueness of the keys as text file is not database and does not enforce uniqueness.

It is good practise to make generators/list comprehensions as they are pythonic and efficient way of doing things. If you do not like them, you can change them to normal loops easy enough.

result = (line.split()
          for  key in set(line.split('\t',1)[0] for line in open('testfile1.txt'))
          for  line in open('testfile2.txt')
          if line.startswith(key+'\t')
          )
for same in sorted(result):
    print(same)

Fantastic!!!!!!!! easy and worked so welll.. ! perfect.

Thank you all!

Notice though that after the loop the result generator is empty. If you need the values many times change result from generator to list comprehension by changing outer () to [].

This question has already been answered. Start a new discussion instead.