I am trying to create a program that reads in a table of temperatures from a csv file and would like to access a a collection of temperatures based on the year and day.
the first column stands for the year the tempratures have been recorded.
the second column stands for a specific day during each month .
the rest of the column represent the temperatures each month.
For example, 2021 - 23 - 119 = 23rd June 2021 has a temperature of 119
Year Day Months from January to December
2018 18 | 45 54 -11 170 99 166 173 177 175 93 74 69
2021 23 | 13 87 75 85 85 119 190 172 156 104 39 53
2020 23 | 63 86 62 128 131 187 163 162 138 104 60 70
So far I have managed to load the data from a CSV File with clojure.data.csv. this returns a sequence of vectors into the program
(defn Load_csv_file [filepath]
(try
(with-open [reader (io/reader filepath)]
(.skip reader 1)
( let [data (csv/read-csv reader)]
(println data) )
)
(catch Exception ex (println (str "LOL Exception: " (.toString ex))))
))
I am currently trying to figure out how to implement this but my reasoning was to create three keys in a map which will take in the year, day and vector of temperatures, to then filter for a specific value.
Any advice on how i can implement this functionality.
Thanks!
i would go with something like this:
(require '[clojure.java.io :refer [reader]]
'[clojure.string :refer [split blank?]]
'[clojure.edn :as edn])
(with-open [r (reader "data.txt")]
(doall (for [ln (rest (line-seq r))
:when (not (blank? ln))
:let [[y d & ms] (mapv edn/read-string (split ln #"\s+\|?\s+"))]]
{:year y :day d :months (vec ms)})))
;;({:year 2018,
;; :day 18,
;; :months [45 54 -11 170 99 166 173 177 175 93 74 69]}
;; {:year 2021,
;; :day 23,
;; :months [13 87 75 85 85 119 190 172 156 104 39 53]}
;; {:year 2020,
;; :day 23,
;; :months [63 86 62 128 131 187 163 162 138 104 60 70]})
by the way, i'm not sure csv format allows different separators (as you have in your example.. anyway this one would work for that)
I would create a map of data that looked something like this
{2020 {23 {:months [63 86 62 128 131 187 163 162 138 104 60 70]}}}
This way you can get the data out in a fairly easy way
(get-in data [2020 23 :months]
So something like this
(->> (Load_csv_file "file.csv")
(reduce (fn [acc [year day & months]] (assoc-in acc [year day] months)) {}))
This will result in the data structure I mentioned now you just need to figure out the location of the data you want
I am trying to read a EMV card using an APDU Command, however it seems the ACR122U external reader is blocking the APDU Command.
Select Command:
APDU-C -> 00 A4 04 00 0E 32 50 41 59 2E 53 59 53 2E 44 44 46 30 31 0E
APDU-R <- Error no response
Is it possible that the ACR122U reader is blocking the command ?
You want to SELECT FILE 1PAY.SYS.DDF01,
"Payment System Environment (PSE)"
To get the PSE directory and the card should response with Application Identifier (AID). but you set the LE=0E replace it to "00" .
Corrected APDU =>
PPSE = '00 a4 04 00 0e 32 50 41 59 2e 53 59 53 2e 44 44 46 30 31 00'
if The selection failed then the ADF doesn't exist (SW1/SW2=6A82)
if the selection is true then Application Identifier (AID) start command
Possible AID's:
A0000000031010
A0000000032020
A0000000041010
A0000000043060
AIDPrefix ='00 a4 04 00 07'
In Om Next, when having data such as:
{:table {:name "Disk Performance Table"
:data [:statistics :performance]}
:chart {:name "Combined Graph"
:data [:statistics :performance]}
:statistics {:performance {:cpu-usage [45 15 32 11 66 44]
:disk-activity [11 34 66 12 99 100]
:network-activity [55 87 20 1 22 82]}}}
you can query it with:
[{:chart [{:data [:cpu-usage]}]}]
to get the chart, join the data and dig down cpu-usage from the performance record:
{:chart {:data {:cpu-usage [45 15 32 11 66 44]}}}
How do I get the whole performance record instead?
Another potential query is this:
[{:chart [:data]}]
but it doesn't resolve the join:
{:chart {:data [:statistics :performance]}}
There are no components as this is only about the data and the query. This is from the exercise number 2 and queries here: https://awkay.github.io/om-tutorial/#!/om_tutorial.D_Queries which uses om/db->tree to run the queries.
This is how you do it:
[{:chart [{:data [*]}]}]
which gives you:
{:chart {:data {:cpu-usage [45 15 32 11 66 44]
:disk-activity [11 34 66 12 99 100]
:network-activity [55 87 20 1 22 82]}}}
Without seeing the actual components with queries and idents, I can't be sure.
However, you should be able to query for [{:chart [:data]}]. See om/db->tree. Assuming that you have structured your components with the right queries and idents, om/db->tree converts your flat app state into a tree so that your read functions see the following data when called:
{:table {:name "Disk Performance Table"
:data {:cpu-usage [45 15 32 11 66 44]
:disk-activity [11 34 66 12 99 100]
:network-activity [55 87 20 1 22 82]}}
:chart {:name "Combined Graph"
:data {:cpu-usage [45 15 32 11 66 44]
:disk-activity [11 34 66 12 99 100]
:network-activity [55 87 20 1 22 82]}}}
If that query doesn't work, [{:chart [{:data [:cpu-usage :disk-activity :network-activity]}]}] should certainly do the trick.
This is the input file input.awk DOS type
06-13-2014,08:43:11
RLS007817
RRC001021
yes,71.61673,0,150,37,1
no,11,156,1.35,306.418
4,3,-1,2.5165,20,-1.4204
-4,0,11,0,0,0
1.00E-001,0.2,3.00E-001,0.6786031,0.5,6.37E-002
110,40,30,222,200,-539
120,50,35,215,220,-547
130,60,40,207,240,-553
140,70,45,196,260,-560
150,80,50,184,280,-566
160,90,55,170,300,-573
170,100,60,157,320,-578
180,110,65,141,340,-582
190,120,70,126,360,-586
200,130,75,110,380,-590
This is what I basically need:
Ignore the first 8 lines (OK)
Pick and split the numbers on lines 6,7 & 8 (OK)
Do AWK math on columns (Error only in first line?)
BASH code
#!/bin/bash
myfile="input.awk"
vzeros=$(sed '6q;d' $myfile)
vshift=$(sed '7q;d' $myfile)
vcalib=$(sed '8q;d' $myfile)
IFS=','
read -a avz <<< "${vzeros}"
read -a avs <<< "${vshift}"
read -a avc <<< "${vcalib}"
z1=${avz[0]};s1=${avs[0]};c1=${avc[0]}
z2=${avz[1]};s2=${avs[1]};c2=${avc[1]}
z3=${avz[2]};s3=${avs[2]};c3=${avc[2]}
z4=${avz[4]};s4=${avs[4]};c4=${avc[4]}
#The single variables will be passed to awk
awk -v z1="$z1" -v c1="$c1" -v s1="$s1" -v z2="$z2" -v c2="$c2" -v s2="$s2" -v z3="$z3" -v c3="$c3" -v s3="$s3" -v z4="$z4" -v c4="$c4" -v s4="$s4" 'NR>8 { FS = "," ;
nc1 = c1 * ( $1 - z1 - s1 );
nc2 = c2 * ( $2 - z2 - s2 );
nc3 = c3 * ( $3 - z3 - s3 );
nc4 = c4 * ( $5 - z4 - s4 );
print nc1,nc2,nc3,nc4 }' $myfile > test.plot
This is the result on the file test.plot
11 -0.6 -3 -10
12 9.4 7.5 100
13 11.4 9 110
14 13.4 10.5 120
15 15.4 12 130
16 17.4 13.5 140
17 19.4 15 150
18 21.4 16.5 160
19 23.4 18 170
20 25.4 19.5 180
This is the weird part... Only in the first line and after the first column all is wrong... And I have no idea why.
This is the expected result file:
11 7.4 6 90
12 9.4 7.5 100
13 11.4 9 110
14 13.4 10.5 120
15 15.4 12 130
16 17.4 13.5 140
17 19.4 15 150
18 21.4 16.5 160
19 23.4 18 170
20 25.4 19.5 180
I've printed the correction factors captured from lines 6,7 & 8 and everything is fine. All math is fine, except on the first line, after the first column.
OS: Slackware 13.37.
AWK: GNU Awk 3.1.6 Copyright (C) 1989, 1991-2007 Free Software Foundation.
I agree with #jeanrjc.
I copied your file and script to my machine and reduced it to processing the first 2 lines of your data.
With your code as is, I duplicate your results, i.e.
#dbg $0=110,40,30,222,200,-539
#dbg c2=0.2 $2= z2=3 s2=0
11 -0.6 -3 -10
#dbg $0=120,50,35,215,220,-547
#dbg c2=0.2 $2= z2=3 s2=0
12 -0.6 -3 -10
With FS=","; commented out, and -F, added in the option list the output is what you are looking for.
#dbg $0=110,40,30,222,200,-539
#dbg c2=0.2 $2=40 z2=3 s2=0
11 7.4 6 90
#dbg $0=120,50,35,215,220,-547
#dbg c2=0.2 $2=50 z2=3 s2=0
12 9.4 7.5 100
So make sure you have removed the FS=","; from the block of code, and you are using -F, In any case, I would say, that resetting the FS="," for each line that is processed is not useful.
If that still doesn't solve it, try the corrected code on a machine with a newer version of awk.
It would take a small magazine article to completely illustrate what is happening while reading thru the first 8 records (when FS="[[:space:]]), the transition to the first row that meets your rule NR>8, the FS is still [:space:] when the fields are parsed, then, FS is set to ,, but that first row is not rescanned.
IHTH!
Your sample is too complex to reproduce something, but I guess you should try :
awk -F"," 'NR>8{...
instead of
awk 'NR>8 { FS = "," ;
You can also try with BEGIN:
awk 'BEGIN{FS=","}NR>8{...
I eventually tested your script, and you should change the position of the FS parameter, as I told you:
awk -v z1="$z1" -v c1="$c1" -v s1="$s1" -v z2="$z2" \
-v c2="$c2" -v s2="$s2" -v z3="$z3" -v c3="$c3" \
-v s3="$s3" -v z4="$z4" -v c4="$c4" -v s4="$s4" -F"," 'NR>8 {
nc1 = c1 * ( $1 - z1 - s1 );
nc2 = c2 * ( $2 - z2 - s2 );
nc3 = c3 * ( $3 - z3 - s3 );
nc4 = c4 * ( $5 - z4 - s4 );
print nc1,nc2,nc3,nc4 }' $myfile
11 7.4 6 90
12 9.4 7.5 100
13 11.4 9 110
14 13.4 10.5 120
15 15.4 12 130
16 17.4 13.5 140
17 19.4 15 150
18 21.4 16.5 160
19 23.4 18 170
20 25.4 19.5 180
0 -0.6 -3 -10
Why you had a problem ?
Because awk parses the line before executing the block, so if you tell it to change something related to parsing, the changes will occur from the next line.
HTH
I have data that goes back to 2004 to deal with so have to simplify calculations moving from using Excel to using SQL to save processing time & pressure on our servers.
My data is:
Period Employee EmOrg EmType Total Hours Mode
201306 GOVINP1 RSA/PZB/T00 S 180 66
201306 LANDCJ1 RSA/PZB/T00 S 200 35
201306 WOODRE RSA/PZB/T00 S 180 34
201306 MOKOHM1 RSA/JNB/T00 S 160 33
201306 KAPPPJ RSA/PLZ/T00 S 160 32
201306 CAHISJ RSA/PZB/T00 S 187 31
201306 ZEMUN RSA/PZB/T00 S 180 31
201306 SAULDD1 RSA/PZB/T00 S 190 28
201306 JEROP1 RSA/DUR/T00 S 188 26
201306 NGOBS1 RSA/PZB/T00 S 204 24
201306 ZONDNS2 RSA/PZB/T00 S 192 23
201306 DLAMMP RSA/PZB/T00 S 201 23
201306 MPHURK RSA/PLZ/T00 S 160 22
201306 MNDAMB RSA/PZB/T00 S 188 21
My desired outcome is:
Period EmOrg EmType TotalHours FTE S
201308 RSA/BFN/T00 S 198 1
201308 RSA/CPT/T00 S 744 3.757575
201308 RSA/DUR/T00 S 805 4.065656
201308 RSA/JNB/T00 S 396 2
201308 RSA/PLZ/T00 S 563 2.843434
201308 RSA/PTA/T00 S 594 3
201308 RSA/PZB/T00 S 4882 24.656565
And my query:
SELECT
LD.Period,
LD.EmOrg,
LD.EmType,
Sum(LD.RegHrs) AS 'Total Hours',
Sum(LD.RegHrs) / 198 As 'FTE_S'
FROM
SSI.dbo.LD LD
GROUP BY LD.Period , LD.EmOrg , LD.EmType
HAVING (LD.EmOrg Like '%T00')
AND (LD.EmType = 'S')
How do I refer to a column in a different worksheet to use as my Mode rather than dividing with an actual number? Because different months have a different mode and using an actual number will give wrong output in other months.
You need to create a separate table for your Mode per month and than use JOIN to get that value and use it.
Something like this:
SELECT
LD.Period,
LD.EmOrg,
LD.EmType,
Sum(LD.RegHrs) AS 'Total Hours',
Sum(LD.RegHrs) / M.Mode As 'FTE_S'
FROM
SSI.dbo.LD LD
INNER JOIN SSI.dbo.Mode M
ON LD.Period = M.Period -- Not sure its should be Period or Month
GROUP BY LD.Period , LD.EmOrg , LD.EmType
HAVING (LD.EmOrg Like '%T00')
AND (LD.EmType = 'S')