forked from ddhostallero/BiG-DRP
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathpreprocess.log
executable file
·129 lines (127 loc) · 13.8 KB
/
preprocess.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
Importing candle utils for pytorch
model name: BiG-DRP
Params:
{'activation': 'ANL',
'auc_threshold': 0.5,
'batch_size': 200,
'binary_file': 'binary_response.csv',
'ckpt_checksum': False,
'ckpt_directory': './save',
'ckpt_keep_limit': 1000000,
'ckpt_keep_mode': 'linear',
'ckpt_restart_mode': 'auto',
'ckpt_save_best': True,
'ckpt_save_best_metric': 'val_loss',
'ckpt_save_interval': 0,
'ckpt_save_weights_only': False,
'ckpt_skip_epochs': 0,
'cuda_name': 'cuda:0',
'data_bin_cleaned_out': 'BiG_DRP_data_bined.csv',
'data_cleaned_out': 'BiG_DRP_data_cleaned.csv',
'data_dir': '/homes/ac.rgnanaolivu/improve_data_dir/BiG-DRP/Data',
'data_file': 'ln_ic50.csv',
'data_tuples_out': 'BiG_DRP_data_tuples.csv',
'data_type': 'CTRPv2,CCLE,GDSCv1,GDSCv2,gCSI',
'data_url': 'http://chia.team/IMPROVE_data/BiG_DRP_data.tar.gz',
'dataroot': '/homes/ac.rgnanaolivu/improve_data_dir/BiG-DRP/',
'descriptor_out': 'BiG_DRP_descriptors.csv',
'drug_feature': 'desc',
'drug_feature_dir': 'drug_feature/',
'drug_response_dir': 'drug_response/',
'drug_synonyms': 'drug_synonyms.txt',
'drugset': 'drug_list.txt',
'epochs': 20,
'experiment_id': 'EXP000',
'expression_dir': 'sanger_tcga/',
'expression_out': 'BiG_DRP_fpkm.csv',
'folder': 'results',
'improve_analysis': 'no',
'improve_data_url': 'https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data',
'labels': 'BiG_DRP_tuple_labels_folds.csv',
'learning_rate': 0.0001,
'logfile': None,
'metric': 'auc',
'mode': 'collate',
'model_name': 'BiG-DRP',
'morgan_out': 'BiG_DRP_morgan.csv',
'network_percentile': 1,
'normalize_response': 'True',
'original_data': 'BiG_DRP_data.tar.gz',
'output_dir': '/homes/ac.rgnanaolivu/improve_data_dir/BiG-DRP/results/EXP000/RUN000',
'outroot': '.',
'profiling': False,
'response': 'auc',
'results_dir': 'results',
'rng_seed': 7102,
'run_id': 'RUN000',
'seed': 0,
'shuffle': False,
'smiles_file': 'BiG_DRP_smiles.csv',
'split': 'lco',
'tcga_file': 'tcga_one_sample.csv',
'timeout': -1,
'train_bool': True,
'train_data_type': 'CTRPv2',
'tuples_label_fold_out': 'BiG_DRP_tuples_fold.csv',
'verbose': False,
'weight_folder': ''}
data downloaded dir is /homes/ac.rgnanaolivu/improve_data_dir/BiG-DRP/csa_data/raw_data
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/CTRPv2_all.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/CCLE_all.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/GDSCv1_all.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/GDSCv2_all.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/gCSI_all.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/CTRPv2_split_0_test.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/CCLE_split_0_test.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/GDSCv1_split_0_test.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/GDSCv2_split_0_test.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/gCSI_split_0_test.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/CTRPv2_split_0_train.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/CCLE_split_0_train.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/GDSCv1_split_0_train.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/GDSCv2_split_0_train.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/gCSI_split_0_train.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/CTRPv2_split_0_val.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/CCLE_split_0_val.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/GDSCv1_split_0_val.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/GDSCv2_split_0_val.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/splits/gCSI_split_0_val.txt
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/x_data/cancer_gene_expression.tsv
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/x_data/drug_SMILES.tsv
downloading file: https://ftp.mcs.anl.gov/pub/candle/public/improve/benchmarks/single_drug_drp/benchmark-data-pilot1/csa_data/raw_data/x_data/drug_ecfp4_nbits512.tsv
Response data: (286665, 4)
improve_sample_id 720
improve_chem_id 494
dtype: int64
Response data: (28666, 4)
improve_sample_id 719
improve_chem_id 494
dtype: int64
Response data: (951, 4)
improve_sample_id 371
improve_chem_id 24
dtype: int64
Response data: (17194, 4)
improve_sample_id 546
improve_chem_id 294
dtype: int64
Response data: (11464, 4)
improve_sample_id 469
improve_chem_id 168
dtype: int64
Response data: (494, 4)
improve_sample_id 258
improve_chem_id 16
dtype: int64
SMILES data: (1565, 2)
smiles file downloaded and reformatted using improve utils /homes/ac.rgnanaolivu/improve_data_dir/BiG-DRP/Data/BiG_DRP_data/BiG_DRP_smiles.csv /homes/ac.rgnanaolivu/improve_data_dir/BiG-DRP/Data/BiG_DRP_data//drug_synonyms.txt
downloading file: http://chia.team/IMPROVE_data/BiG_DRP_data.tar.gz
Gene expression data: (1007, 30805)
/homes/ac.rgnanaolivu/improve_data_dir/BiG-DRP/Data/BiG_DRP_data//ln_ic50.csv
/homes/ac.rgnanaolivu/improve_data_dir/BiG-DRP/Data/BiG_DRP_data//binary_response.csv
/homes/ac.rgnanaolivu/improve_data_dir/BiG-DRP/Data/BiG_DRP_data//drug_synonyms.txt
/homes/ac.rgnanaolivu/improve_data_dir/BiG-DRP/Data/BiG_DRP_data//drp-data/grl-preprocessed/sanger_tcga//BiG_DRP_fpkm.csv
Data matrix size: (494, 720)
Processing binarized data...
BIN: cells match? False
KeyError: "['ACH-000002', 'ACH-000005', 'ACH-000006', 'ACH-000007', 'ACH-000008', 'ACH-000011', 'ACH-000012', 'ACH-000018', 'ACH-000022', 'ACH-000023', 'ACH-000027', 'ACH-000028', 'ACH-000032', 'ACH-000035', 'ACH-000038', 'ACH-000039', 'ACH-000040', 'ACH-000041', 'ACH-000046', 'ACH-000050', 'ACH-000051', 'ACH-000055', 'ACH-000058', 'ACH-000059', 'ACH-000061', 'ACH-000065', 'ACH-000066', 'ACH-000068', 'ACH-000069', 'ACH-000070', 'ACH-000076', 'ACH-000077', 'ACH-000078', 'ACH-000080', 'ACH-000081', 'ACH-000082', 'ACH-000086', 'ACH-000087', 'ACH-000090', 'ACH-000091', 'ACH-000093', 'ACH-000097', 'ACH-000098', 'ACH-000099', 'ACH-000102', 'ACH-000105', 'ACH-000106', 'ACH-000110', 'ACH-000112', 'ACH-000113', 'ACH-000115', 'ACH-000116', 'ACH-000117', 'ACH-000122', 'ACH-000123', 'ACH-000124', 'ACH-000126', 'ACH-000129', 'ACH-000132', 'ACH-000133', 'ACH-000135', 'ACH-000136', 'ACH-000137', 'ACH-000140', 'ACH-000141', 'ACH-000142', 'ACH-000146', 'ACH-000147', 'ACH-000150', 'ACH-000151', 'ACH-000152', 'ACH-000154', 'ACH-000156', 'ACH-000157', 'ACH-000158', 'ACH-000159', 'ACH-000161', 'ACH-000162', 'ACH-000163', 'ACH-000164', 'ACH-000169', 'ACH-000171', 'ACH-000173', 'ACH-000174', 'ACH-000176', 'ACH-000177', 'ACH-000178', 'ACH-000180', 'ACH-000181', 'ACH-000182', 'ACH-000186', 'ACH-000188', 'ACH-000189', 'ACH-000190', 'ACH-000191', 'ACH-000194', 'ACH-000195', 'ACH-000197', 'ACH-000200', 'ACH-000201', 'ACH-000203', 'ACH-000209', 'ACH-000210', 'ACH-000211', 'ACH-000212', 'ACH-000218', 'ACH-000220', 'ACH-000221', 'ACH-000225', 'ACH-000226', 'ACH-000228', 'ACH-000232', 'ACH-000235', 'ACH-000236', 'ACH-000237', 'ACH-000238', 'ACH-000239', 'ACH-000241', 'ACH-000243', 'ACH-000244', 'ACH-000245', 'ACH-000249', 'ACH-000250', 'ACH-000253', 'ACH-000255', 'ACH-000258', 'ACH-000260', 'ACH-000261', 'ACH-000264', 'ACH-000266', 'ACH-000267', 'ACH-000269', 'ACH-000273', 'ACH-000274', 'ACH-000278', 'ACH-000280', 'ACH-000285', 'ACH-000287', 'ACH-000290', 'ACH-000292', 'ACH-000294', 'ACH-000295', 'ACH-000296', 'ACH-000298', 'ACH-000301', 'ACH-000302', 'ACH-000303', 'ACH-000305', 'ACH-000306', 'ACH-000308', 'ACH-000310', 'ACH-000313', 'ACH-000316', 'ACH-000317', 'ACH-000318', 'ACH-000321', 'ACH-000322', 'ACH-000323', 'ACH-000324', 'ACH-000325', 'ACH-000326', 'ACH-000327', 'ACH-000332', 'ACH-000335', 'ACH-000336', 'ACH-000338', 'ACH-000343', 'ACH-000344', 'ACH-000346', 'ACH-000349', 'ACH-000352', 'ACH-000353', 'ACH-000357', 'ACH-000358', 'ACH-000360', 'ACH-000362', 'ACH-000363', 'ACH-000368', 'ACH-000369', 'ACH-000370', 'ACH-000372', 'ACH-000376', 'ACH-000377', 'ACH-000387', 'ACH-000388', 'ACH-000390', 'ACH-000394', 'ACH-000396', 'ACH-000398', 'ACH-000399', 'ACH-000402', 'ACH-000403', 'ACH-000404', 'ACH-000407', 'ACH-000408', 'ACH-000410', 'ACH-000415', 'ACH-000416', 'ACH-000417', 'ACH-000418', 'ACH-000419', 'ACH-000428', 'ACH-000432', 'ACH-000433', 'ACH-000434', 'ACH-000437', 'ACH-000438', 'ACH-000439', 'ACH-000441', 'ACH-000445', 'ACH-000451', 'ACH-000452', 'ACH-000453', 'ACH-000454', 'ACH-000458', 'ACH-000459', 'ACH-000460', 'ACH-000461', 'ACH-000464', 'ACH-000466', 'ACH-000467', 'ACH-000469', 'ACH-000471', 'ACH-000472', 'ACH-000479', 'ACH-000484', 'ACH-000485', 'ACH-000486', 'ACH-000487', 'ACH-000488', 'ACH-000489', 'ACH-000491', 'ACH-000495', 'ACH-000496', 'ACH-000500', 'ACH-000501', 'ACH-000502', 'ACH-000503', 'ACH-000504', 'ACH-000506', 'ACH-000509', 'ACH-000511', 'ACH-000513', 'ACH-000514', 'ACH-000515', 'ACH-000517', 'ACH-000518', 'ACH-000519', 'ACH-000521', 'ACH-000523', 'ACH-000527', 'ACH-000530', 'ACH-000532', 'ACH-000534', 'ACH-000537', 'ACH-000542', 'ACH-000543', 'ACH-000545', 'ACH-000546', 'ACH-000547', 'ACH-000549', 'ACH-000550', 'ACH-000553', 'ACH-000557', 'ACH-000560', 'ACH-000563', 'ACH-000566', 'ACH-000567', 'ACH-000568', 'ACH-000570', 'ACH-000574', 'ACH-000576', 'ACH-000578', 'ACH-000579', 'ACH-000587', 'ACH-000592', 'ACH-000593', 'ACH-000598', 'ACH-000599', 'ACH-000602', 'ACH-000603', 'ACH-000604', 'ACH-000605', 'ACH-000606', 'ACH-000607', 'ACH-000608', 'ACH-000609', 'ACH-000613', 'ACH-000615', 'ACH-000616', 'ACH-000617', 'ACH-000619', 'ACH-000623', 'ACH-000626', 'ACH-000630', 'ACH-000632', 'ACH-000635', 'ACH-000636', 'ACH-000639', 'ACH-000640', 'ACH-000641', 'ACH-000645', 'ACH-000647', 'ACH-000657', 'ACH-000661', 'ACH-000670', 'ACH-000672', 'ACH-000678', 'ACH-000680', 'ACH-000681', 'ACH-000683', 'ACH-000685', 'ACH-000686', 'ACH-000688', 'ACH-000689', 'ACH-000692', 'ACH-000694', 'ACH-000695', 'ACH-000697', 'ACH-000698', 'ACH-000699', 'ACH-000702', 'ACH-000703', 'ACH-000705', 'ACH-000706', 'ACH-000707', 'ACH-000709', 'ACH-000713', 'ACH-000714', 'ACH-000715', 'ACH-000716', 'ACH-000718', 'ACH-000720', 'ACH-000721', 'ACH-000723', 'ACH-000725', 'ACH-000726', 'ACH-000728', 'ACH-000730', 'ACH-000731', 'ACH-000732', 'ACH-000733', 'ACH-000735', 'ACH-000736', 'ACH-000740', 'ACH-000741', 'ACH-000743', 'ACH-000744', 'ACH-000746', 'ACH-000748', 'ACH-000751', 'ACH-000752', 'ACH-000753', 'ACH-000754', 'ACH-000758', 'ACH-000762', 'ACH-000763', 'ACH-000764', 'ACH-000765', 'ACH-000767', 'ACH-000768', 'ACH-000770', 'ACH-000771', 'ACH-000772', 'ACH-000773', 'ACH-000774', 'ACH-000778', 'ACH-000780', 'ACH-000782', 'ACH-000785', 'ACH-000789', 'ACH-000790', 'ACH-000791', 'ACH-000793', 'ACH-000794', 'ACH-000796', 'ACH-000800', 'ACH-000801', 'ACH-000802', 'ACH-000803', 'ACH-000806', 'ACH-000807', 'ACH-000808', 'ACH-000813', 'ACH-000815', 'ACH-000816', 'ACH-000821', 'ACH-000823', 'ACH-000827', 'ACH-000828', 'ACH-000831', 'ACH-000833', 'ACH-000834', 'ACH-000835', 'ACH-000836', 'ACH-000837', 'ACH-000839', 'ACH-000840', 'ACH-000841', 'ACH-000843', 'ACH-000844', 'ACH-000845', 'ACH-000851', 'ACH-000852', 'ACH-000853', 'ACH-000855', 'ACH-000856', 'ACH-000857', 'ACH-000862', 'ACH-000863', 'ACH-000868', 'ACH-000869', 'ACH-000874', 'ACH-000876', 'ACH-000879', 'ACH-000880', 'ACH-000882', 'ACH-000888', 'ACH-000889', 'ACH-000890', 'ACH-000891', 'ACH-000895', 'ACH-000896', 'ACH-000897', 'ACH-000898', 'ACH-000899', 'ACH-000904', 'ACH-000905', 'ACH-000909', 'ACH-000912', 'ACH-000913', 'ACH-000915', 'ACH-000917', 'ACH-000918', 'ACH-000920', 'ACH-000922', 'ACH-000923', 'ACH-000924', 'ACH-000925', 'ACH-000929', 'ACH-000930', 'ACH-000931', 'ACH-000932', 'ACH-000934', 'ACH-000935', 'ACH-000937', 'ACH-000939', 'ACH-000941', 'ACH-000942', 'ACH-000944', 'ACH-000947', 'ACH-000951', 'ACH-000953', 'ACH-000955', 'ACH-000956', 'ACH-000957', 'ACH-000959', 'ACH-000960', 'ACH-000963', 'ACH-000964', 'ACH-000965', 'ACH-000967', 'ACH-000968', 'ACH-000970', 'ACH-000973', 'ACH-000978', 'ACH-000981', 'ACH-000982', 'ACH-000983', 'ACH-000985', 'ACH-000986', 'ACH-000989', 'ACH-000990', 'ACH-000991', 'ACH-000993', 'ACH-000994', 'ACH-000995', 'ACH-001129', 'ACH-001306'] not in index"