How to code a complex array of hashes in Perl? - csv

Here is a task that's puzzling me for quite a while now:
Regularly, we get exam results of our students as a .csv file. The header has some meta data such as ID, gender, date of birth, status, exam room, seat number, exam version, and the score for 60 questions (0.0 0.5 1.0 points if the answer is wrong, half-correct, or correct). There are 6 versions (A - F) of the exam differing only by the order of the 60 questions. The information is stored for statistical evaluation which requires the correct alignment according to the exam master (a .txt file with 7 columns for version A-F and the correct answer in the 7th column).
I tried to accommodate the .csv file as an array of hashes to generate a different .csv or tabbed .txt file in which all exam results appear in a unified order for later statistical evaluation.
Example:
header --
ID,gender,birthdate,order,room,seat,version,points,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
277710814533,f,01/02/1993,m,sr_3,A11,A, 1,1,1,1,0,1,1,1,.5,1,1,1,0,1,.5,1,1,1,0,1,.5,1,1,0,1,1,1,1,1,1,1,0,0,1,0,1,.5,1,1,1,1,.5,0,1,1,1,0,1,1,1,1,1,0,1,1,1,.5,1,1,1
755310765962,f,31/07/1992 00:00,v,aula,C11,C,1,.5,0,1,1,1,1,1,1,1,1,1,1,1,0,1,1,1,1,1,0,1,1,1,1,0,1,1,1,.5,1,0,.5,1,0,1,.5,0,.5,0,1,0,0,.5,1,1,0,.5,1,1,.5,.5,1,.5,.5,1,1,1,.5,.5
394610513538,m,20/10/1992 00:00,m,sr_3,E13,E,1,1,0,.5,1,1,1,1,1,1,1,.5,1,1,.5,.5,1,1,1,.5,.5,1,1,1,1,0,0,.5,1,1,.5,.5,.5,.5,0,1,0,.5,0,0,1,0,1,.5,0,1,0,0,.5,1,0,1,1,0,.5,.5,.5,.5,.5,.5
The code generates hash keys according to the following scheme:
while ( <FH> ) {
chomp ;
if ( /^\d\d\d/) {
( $id , $gender , $birthday , $status , $room , $seat , $version , #points ) = split ( /,/ , $_ ) ;
$student = {
'id' => $id ,
'gender' => $gender ,
'birthday' => $birthday ,
'position' => $position ,
'room' => $room ,
'seat' => $seat ,
'version' => $version ,
'points' => #points
} ;
push ( #candidates , $student ) ;
}
} ;
close FH ;
print "Number of candidates processed: " . ( $#candidates + 1 ) . "\n" ;
The compiler throws a warning for each record, e.g. "Odd number of elements in anonymous hash at /Documents//testAoH.pl line 38, line 16." but the script is executed.
The script prints the correct number of processed records, but when I try to retrieve a specific record I only get the scalar values and the #points array yields only one (the first?) result as if it were destroyed. A data dumper output further shows that something must be internally wrong with this code.
Data Dumper e.g.
755310765962
$VAR1 = \{
'0' => '0',
'gender' => 'f',
'id' => '755310765962',
'points' => '1',
'room' => 'aula',
'.5' => undef,
'1' => '.5',
'birthday' => '31/07/1992',
'seat' => 'A11',
'version' => 'A',
'status' => 'v'
};
Any clues?

Use \#points. #points expands in the hash constructor to generate:
'Points' => $points[0], $points[1] => $points[2], ...

Related

MYSQL: How to sort number DESC with plus value as high

I need some help to create desire output with pure MYSQL. My question is By fire SQL below:
SELECT
r.questions_id
,r.Empty_Peg_Count AS 'Empty_Pegs'
FROM
Training_Core.results AS r
ORDER BY
CAST(r.Empty_Peg_Count AS SIGNED INTEGER) DESC;
I'm getting below output:
Array
(
[0] => Array
(
[questions_id] => 256332653
[Empty_Pegs] => 2
)
[1] => Array
(
[questions_id] => 256332653
[Empty_Pegs] => 2+
)
[2] => Array
(
[questions_id] => 256332653
[Empty_Pegs] => 1
)
)
I want value with + sign as high priority and it should show first. my desired output is as below:
Array
(
[0] => Array
(
[questions_id] => 256332653
[Empty_Pegs] => 2+
)
[1] => Array
(
[questions_id] => 256332653
[Empty_Pegs] => 2
)
[2] => Array
(
[questions_id] => 256332653
[Empty_Pegs] => 1
)
)
Can anyone help me how to get this result with SQL query?
Thanks an advance!
You can use a second order by rule:
ORDER BY
CAST(r.Empty_Peg_Count AS SIGNED INTEGER) DESC
,r.Empty_Peg_Count LIKE '%+%' DESC;
This will push the entries with a + to the top.
You can use RIGHT to check if the last character is a + and use that as a secondary ORDER BY field:
SELECT
r.questions_id
,r.Empty_Peg_Count AS 'Empty_Pegs'
FROM
Training_Core.results AS r
ORDER BY
CAST(r.Empty_Peg_Count AS SIGNED INTEGER) DESC
, RIGHT(r.Empty_Peg_Count, 1) = '+' DESC
You can apply sort using the following
SELECT * FROM `data` ORDER BY
CASE
WHEN right(`Empty_Pegs`,1) = '+' THEN (left(`Empty_Pegs`,1)*10)
ELSE `Empty_Pegs`
END DESC;

Cakephp Table query: SUM returning 0

I'm having problems when using find to sum the field num_days from table events.
$events = TableRegistry::get('Events');
$query = $events->find('all')
->select(['used' => 'sum(num_days)'])
->first();
Why $query->used is always 0?
print_r($events->find('all')->select(['used' => 'sum(num_days)'])->toArray()) gives,
Array ( [0] => App\Model\Entity\Event Object ( [used] => 6 [[new]] => [[accessible]] => Array ( [*] => 1 ) [[dirty]] => Array ( ) [[original]] => Array ( ) [[virtual]] => Array ( ) [[errors]] => Array ( ) [[invalid]] => Array ( ) [[repository]] => Events ) )
6 is exactly the correct answer for the query and print_r shows it but $query->used is returning always 0.
try $query->order(['used' => 'DESC']); before $query->used
Also, we can add used as protected $_virtual = ['used']; inside Event.php Entity File

Getting Error when using POST Orders on Shopify

I'm trying to post a new order which comes from another marketplace. I have all the correct information, but for some reason the POST is not accepting or recognizing the variant_id Im sending. It always returns an error saying I'm missing name, price, and title. I'm not sure why this comes. Here is the array Im sending:
Array
(
[order] => Array
(
[line_items] => Array
(
[0] => Array
(
[id] => 2147483647
[quantity] => 1
[price] => 609
)
)
[customer] => Array
(
[first_name] => Jurgen
[last_name] => Feuchter Garcia
[email] => mail#gmail.com
)
[note] => /// ***** ORDEN MERCADOLIBRE ***** /// ID DE ORDEN ML: order_number
[financial_status] => pending
[tags] => Array
(
[0] => Orden MercadoLibre
)
)
)
And here is the response:
Array
(
[errors] => Array
(
[order] => Array
(
[0] => Line items is invalid
)
[line_items] => Array
(
[0] => Name can't be blank
[1] => Title can't be blank
)
)
)
I've tried using the value name variant_id instead of id, and it asked me for the same information. Any ideas why this might be happening?
I actually found out what was wrong, the variant_id was wrong. Apparently if the variant_id is wrong, it tells you that name and title is missing, and if you have it right, it mentions that price line is missing. This is what happened to me. Not totally sure if it works like that, but I got it to work using the correct variant_id and adding pricing for the product.
You can't add a price like that and mind that the id pertains to a variant id and not the product id.
Your variant with id 214748364 should already be priced before hand in the Shopify admin dashboard (backend).
Error 1 is displayed because Price is not part of line_items.
Error 2 is however strange because, Name and Title are not compulsory parts of line_items. For e.g. below example would create an order successfully.
POST /admin/orders.json
{
"order": {
"email": "foo#example.com",
"fulfillment_status": "fulfilled",
"send_receipt": true,
"send_fulfillment_receipt": true,
"line_items": [
{
"variant_id": 447654529,
"quantity": 1
}
]
}
}

Cakephp find('list') with summarized data returns nulls

I have the following query in a Cakephp 2.4 model:
$scores = $this->WorkInfo->find('list', array(
'conditions' => array('WorkInfo.work_id' => $work_ids),
'fields' => array('WorkInfo.date', 'SUM(WorkInfo.score)'),
'group' => array('WorkInfo.date')
));
Which generates the following query:
SELECT
`WorkInfo`.`date`,
SUM(`WorkInfo`.`score`)
FROM
`home`.`work_infos` AS `WorkInfo`
WHERE
`WorkInfo`.`work_id` IN (4, 7, 8, 12, 9, 11, 13, 10, 14, 6, 5)
GROUP BY
`WorkInfo`.`date`
The result I get in my application is:
'2014-03-24' => null
'2014-03-25' => null
'2014-03-26' => null
'2014-03-27' => null
'2014-03-28' => null
'2014-03-31' => null
While the result I get from pasting this very query in the mysql console is:
'2014-03-24' => 0
'2014-03-25' => 36
'2014-03-26' => 0
'2014-03-27' => 164
'2014-03-28' => 0
'2014-03-31' => 0
What is going on here? It is supposed that same queries output same results, isn't it?
I have read something about creating virtual fields for this, but I do not want to overkill, it should be possible to perform a simple aggregation query through Cakephp using the find function.
Thanks!
Try this
$scores = $this->WorkInfo->find('all', array(
'conditions' => array('work_id' => $work_ids),
'fields' => array('date', 'SUM(score) AS score'),
'group' => array('date')
));
then with Set::combine you can format your array cakephp find list
$scores = Set::combine($scores, '{n}.WorkInfo.date', '{n}.0.score');
prints=>
'2014-03-24' => 0
'2014-03-25' => 36
'2014-03-26' => 0
'2014-03-27' => 164
'2014-03-28' => 0
'2014-03-31' => 0
Ok, sadly, I think what you want to do can't be done as you want to do it.
Let's see, you use the find('list') method, so that's here in the API. Code looks normal, and as you said, query is ok, returns everything you want. Problem is in line 2883
return Hash::combine($results, $query['list']['keyPath'], $query['list']['valuePath'], $query['list']['groupPath']);
That line organizes the returned array after the query is done. And seeing the doc for that function, we have
Creates an associative array using a $keyPath as the path to build its
keys, and optionally $valuePath as path to get the values. If
$valuePath is not specified, or doesn’t match anything, values will be
initialized to null.
Which is what happens to you. Now, debugging, the query result before applying the Hash::combine function is something like this
Array
(
[0] => Array
(
[WorkInfo] => Array
(
[date] => 2013-04-01
)
[0] => Array
(
[SUM(`WorkInfo`.`score`)] => 24
)
)
)
so you see, you get the results. And the respective Hash::combine
Array
(
[groupPath] =>
[valuePath] => {n}.SUM(WorkInfo.score)
[keyPath] => {n}.WorkInfo.date
)
which probably causes problem with the dot inside the parenthesis. And the combine function doesn't find the valuePath, and you get null, and you get sad.
If you change your query to 'SUM(WorkInfo.score) AS score' (leaving everything as is), you have almost the same problem with valuePath
Array
(
[groupPath] =>
[valuePath] => {n}.SUM(WorkInfo.score) as score
[keyPath] => {n}.WorkInfo.date
)
//respective result array
Array
(
[0] => Array
(
[WorkInfo] => Array
(
[date] => 2013-04-01
)
[0] => Array
(
[score] => 24
)
)
)
You might think that doing 'SUM(score) AS score' (without the dot) will solve it, but the code of find('list') adds the alias if it doesn't find a dot (in line 2865).
So... I guess what I'm saying is: do a virtual field, or listen to Isaac Rajaei, or create a custom find function. But with find('list') and SUM() you won't have luck :(

How to Group by all 12 months in MySQL?

I am using cakephp 1.3. I have date field in incomes table. I am trying to get total income per month in a loop. The following is my query.
$income_and_hst = $Income->find('all', array(
'fields' => array('SUM(Income.amount) as income',
'SUM(Income.hst_amount) as hst',
'MONTH(date) as month'),
'conditions' => array(
'Income.income_account_id' => array(1,2,3,4),
'Income.user_id' => $this->Auth->user('id'),
'Income.account_id' => $this->Session->read('Account.default_account'),
'Income.date >' => $starting_date,
'Income.date <' => $ending_date,
),
'group' => 'MONTH(date)',
)
);
This gives me 5 months of income. Because the income were from 5 months. I need to show all 12 months even there is no income in other months. If there is no income i want 0 for that month.
Could someone give me a direction please?
Thank you.
My guess is that you cannot do that using that interface. You either need to dive into raw SQL (and a cross join), or add the missing months in the surrounding php code. I suggest the latter something like this should work (pseudocode, i don't remember php syntax):
for ($row in $income_and_hst) {
$income_and_hst_all[$row['month']] = $row
}
for ($month = 1;$month <= 12; $month++) {
if ($income_and_hst_all[$month] == nil) {
$income_and_hst_all[$month] = Array (
'income' => 0,
'hst' => 0,
'month' => $month
)
}
}