Export a text file to MySQL in Talend - mysql

Question

So I have many files like this :
The first file :
File1;
Code;1971;1981;1991;2001;2011
A;10;20;30;40;50
B;12;22;32;89;95
...
...
The second file :
File2;
Code;1971;1981;1991;2001;2011
A;1500;1600;460;6000;8000
B;6000;7000;8007;8009;9005
...
...
All Files have the exact same format.
I like to have a table in my database like this :
File Code Year Value
File1 A 1971 10
File1 A 1981 20
File1 A 1991 30
. . .
. . .
File2 A 1971 1500
File2 A 1981 1600
File1 A 1991 460
. . . .
. . . .
File2 B 1971 .
File2 B 1981 .
. . . .
My idea is to creat for each file a t_map in wich we have the file as follow :
My solution with t_map
The problem is that I have so many file like this, and my solution will take me a long time to finish it. Is there any better solution ?

Answer 1

I prefer using PHP to dump all data into MySQL...
Code;1971;1981;1991;2001;2011
A;10;20;30;40;50
B;12;22;32;89;95
after Open & Read file ($file), try this...
<?php
function _T($a){return rtrim(trim($a));}
function _E($a,$b){return explode($a,$b);}
function _S($a,$b,$c){return str_replace($a,$b,$c);}
function _G($a,$b,$c){$d=strpos($a,$b);$e=substr($a,$d);$f=strpos($a,$c);return substr($e,0,($f-$d));}
$file='Code;1971;1981;1991;2001;2011
A;10;20;30;40;50
B;12;22;32;89;95';
$fC=_E('|',_S('Code;','|Code;',$file));
/*Useful with Multiple "Code;" in same file too*/
$fC2=($file).'|';
/* Add "|" to the last */
$fY=array();/*Year*/
$fA=array();/*A*/
$fB=array();/*B*/
$dB=array();/* For Database */
foreach($fC as $fR){
/* Get Year; */
$gY=_E(';',_S('Code;','',_G(_T($fR),'Code;','A;')));
$c=0;foreach($gY as $yR){if(!empty($yR)){/*echo _T($yR).'<br>';*/$fY[$c++]=_T($yR);}}
/* Get A */
$gA=_E(';',_S('A;','',_G(_T($fR),'A;','B;')));
$c=0;foreach($gA as $aR){if(!empty($aR)){/*echo _T($aR).'<br>';*/$fA[$c++]=_T($aR);}}
/* Get B */
$gB=_E(';',_S('B;','',_G(_T($fR),'B;','|')));
$c=0;foreach($gB as $bR){if(!empty($bR)){/*echo _T($bR).'<br>';*/$fB[$c++]=_T($bR);}}
if(empty($fB)){$gB=_E(';',_S('B;','',_G(_T($fC2),'B;','|')));
$c=0;foreach($gB as $bR){if(!empty($bR)){/*echo _T($bR).'<br>';*/$fB[$c++]=_T($bR);}}}
}
/* Show Result */
foreach($fY as $fK=>$fV){
echo $fK.':'.$fV.'='.$fA[$fK].'='.$fB[$fK].'<br>';
$dB[]=array('Year'=>$fV,'A'=>$fA[$fK],'B'=>$fB[$fK]);
}print_r($dB);
?>
Test yourself... :)