Using 2 worksheets to calculate in sql - mysql

I have data that goes back to 2004 to deal with so have to simplify calculations moving from using Excel to using SQL to save processing time & pressure on our servers.
My data is:
Period Employee EmOrg EmType Total Hours Mode
201306 GOVINP1 RSA/PZB/T00 S 180 66
201306 LANDCJ1 RSA/PZB/T00 S 200 35
201306 WOODRE RSA/PZB/T00 S 180 34
201306 MOKOHM1 RSA/JNB/T00 S 160 33
201306 KAPPPJ RSA/PLZ/T00 S 160 32
201306 CAHISJ RSA/PZB/T00 S 187 31
201306 ZEMUN RSA/PZB/T00 S 180 31
201306 SAULDD1 RSA/PZB/T00 S 190 28
201306 JEROP1 RSA/DUR/T00 S 188 26
201306 NGOBS1 RSA/PZB/T00 S 204 24
201306 ZONDNS2 RSA/PZB/T00 S 192 23
201306 DLAMMP RSA/PZB/T00 S 201 23
201306 MPHURK RSA/PLZ/T00 S 160 22
201306 MNDAMB RSA/PZB/T00 S 188 21
My desired outcome is:
Period EmOrg EmType TotalHours FTE S
201308 RSA/BFN/T00 S 198 1
201308 RSA/CPT/T00 S 744 3.757575
201308 RSA/DUR/T00 S 805 4.065656
201308 RSA/JNB/T00 S 396 2
201308 RSA/PLZ/T00 S 563 2.843434
201308 RSA/PTA/T00 S 594 3
201308 RSA/PZB/T00 S 4882 24.656565
And my query:
SELECT
LD.Period,
LD.EmOrg,
LD.EmType,
Sum(LD.RegHrs) AS 'Total Hours',
Sum(LD.RegHrs) / 198 As 'FTE_S'
FROM
SSI.dbo.LD LD
GROUP BY LD.Period , LD.EmOrg , LD.EmType
HAVING (LD.EmOrg Like '%T00')
AND (LD.EmType = 'S')
How do I refer to a column in a different worksheet to use as my Mode rather than dividing with an actual number? Because different months have a different mode and using an actual number will give wrong output in other months.

You need to create a separate table for your Mode per month and than use JOIN to get that value and use it.
Something like this:
SELECT
LD.Period,
LD.EmOrg,
LD.EmType,
Sum(LD.RegHrs) AS 'Total Hours',
Sum(LD.RegHrs) / M.Mode As 'FTE_S'
FROM
SSI.dbo.LD LD
INNER JOIN SSI.dbo.Mode M
ON LD.Period = M.Period -- Not sure its should be Period or Month
GROUP BY LD.Period , LD.EmOrg , LD.EmType
HAVING (LD.EmOrg Like '%T00')
AND (LD.EmType = 'S')

Related

is possible to create a map with 2 keys and a vector of values in Clojure?

I am trying to create a program that reads in a table of temperatures from a csv file and would like to access a a collection of temperatures based on the year and day.
the first column stands for the year the tempratures have been recorded.
the second column stands for a specific day during each month .
the rest of the column represent the temperatures each month.
For example, 2021 - 23 - 119 = 23rd June 2021 has a temperature of 119
Year Day Months from January to December
2018 18 | 45 54 -11 170 99 166 173 177 175 93 74 69
2021 23 | 13 87 75 85 85 119 190 172 156 104 39 53
2020 23 | 63 86 62 128 131 187 163 162 138 104 60 70
So far I have managed to load the data from a CSV File with clojure.data.csv. this returns a sequence of vectors into the program
(defn Load_csv_file [filepath]
(try
(with-open [reader (io/reader filepath)]
(.skip reader 1)
( let [data (csv/read-csv reader)]
(println data) )
)
(catch Exception ex (println (str "LOL Exception: " (.toString ex))))
))
I am currently trying to figure out how to implement this but my reasoning was to create three keys in a map which will take in the year, day and vector of temperatures, to then filter for a specific value.
Any advice on how i can implement this functionality.
Thanks!
i would go with something like this:
(require '[clojure.java.io :refer [reader]]
'[clojure.string :refer [split blank?]]
'[clojure.edn :as edn])
(with-open [r (reader "data.txt")]
(doall (for [ln (rest (line-seq r))
:when (not (blank? ln))
:let [[y d & ms] (mapv edn/read-string (split ln #"\s+\|?\s+"))]]
{:year y :day d :months (vec ms)})))
;;({:year 2018,
;; :day 18,
;; :months [45 54 -11 170 99 166 173 177 175 93 74 69]}
;; {:year 2021,
;; :day 23,
;; :months [13 87 75 85 85 119 190 172 156 104 39 53]}
;; {:year 2020,
;; :day 23,
;; :months [63 86 62 128 131 187 163 162 138 104 60 70]})
by the way, i'm not sure csv format allows different separators (as you have in your example.. anyway this one would work for that)
I would create a map of data that looked something like this
{2020 {23 {:months [63 86 62 128 131 187 163 162 138 104 60 70]}}}
This way you can get the data out in a fairly easy way
(get-in data [2020 23 :months]
So something like this
(->> (Load_csv_file "file.csv")
(reduce (fn [acc [year day & months]] (assoc-in acc [year day] months)) {}))
This will result in the data structure I mentioned now you just need to figure out the location of the data you want

Writing multivariate objective function, with matrix data entities

S1=[20 32 44 56 68 80 92 104 116 128 140 152 164 176 188 200];
P=[16.82 26.93 37.01 47.1 57.21 67.32 77.41 87.5 97.54 107.7 117.8 127.9 138 148 158.2 168.3];
X = [0.119 0.191 0.262 0.334 0.405 0.477 0.548 0.620 0.691 0.763 0.835 0.906 0.978 1.049 1.120 1.192];
S = [2.3734 3.6058 5.0256 6.6854 8.6413 10.978 13.897 17.396 21.971 28.040 36.475 49.065 69.736 110.20 224.69 2779.1];
objective=#(x)((1250*x(3)*S(a)-(S(a)+x(2))*(P(a)+x(1)))/(1250*(S(a)+x(2))*(P(a)+x(1)))-x(5))^2+((x(2)*(P(a)^2+x(1)*P(a)))/(1250*x(4)*X(a)*x(3)-P(a)^2-x(1)*P(a))-S(a))^2+(74000/3*((X(a)*x(3)*S(a))/S1(a)*(S(a)+x(2)))-P(a))^2
%x0 = [Kp Ks mu.m Yp mu.d]
x0=[7.347705469 14.88611028 1.19747242 16.65696429 6.01E-03];
x=fminunc(objective,x0);
disp(x)
The code above is used for optimisizing the objective function, so that all the unknown values of the parameters can be found. As you may have seen, the objective function consists of 4 variables (S1, S, P, X), each having 16 data entities. My question is: how to create an objective function, so that all the data entities are utilised?
The final objective function has to be the sum of the objective function (shown above) with a=1:16. Any ideas?
Make the following changes to your code:
Replace, e.g. all S(a) variables with S to use the whole vector. Do the same for each of your four variables.
Convert all 'scalar' operations in your objective function to 'elementwise' ones, i.e. replace ^, * and / with .^, .* and ./. This produces 16 values, one for each index from 1 to 16 (i.e. what was previously referred to by a).
wrap the resulting expression into a sum() function to sum the 16 results into a final value
Use your optimiser as normal.
Resulting code:
S1 = [20 32 44 56 68 80 92 104 116 128 140 152 164 176 188 200];
P = [16.82 26.93 37.01 47.1 57.21 67.32 77.41 87.5 97.54 107.7 117.8 127.9 138 148 158.2 168.3];
X = [0.119 0.191 0.262 0.334 0.405 0.477 0.548 0.620 0.691 0.763 0.835 0.906 0.978 1.049 1.120 1.192];
S = [2.3734 3.6058 5.0256 6.6854 8.6413 10.978 13.897 17.396 21.971 28.040 36.475 49.065 69.736 110.20 224.69 2779.1];
objective = #(x) sum( ((1250.*x(3).*S-(S+x(2)).*(P+x(1)))./(1250.*(S+x(2)).*(P+x(1)))-x(5)).^2+((x(2).*(P.^2+x(1).*P))./(1250.*x(4).*X.*x(3)-P.^2-x(1).*P)-S).^2+(74000./3.*((X.*x(3).*S)./S1.*(S+x(2)))-P).^2 );
%x0 = [Kp Ks mu.m Yp mu.d]
x0 = [7.347705469 14.88611028 1.19747242 16.65696429 6.01E-03];
x = fminunc(objective,x0);
disp(x)
Note that you can make this code a lot clearer to read for humans; I just made the "direct" changes that illustrate conversion from your scalar expression to the desired vectorised one.

Padding preceding blanks on a variable for Money or Disk Space lineup

OK, Basically I am echo'ing a line to a CSV comma delimited.
This is what is happening:
This is the output:
Computer1 Fri 08/04/2017 13:20 110 917 340 907
Computer2 Fri 08/04/2017 13:21 110 917 435 852
Computer3 Fri 08/04/2017 12:39 180 92 916
Computer4 Fri 08/04/2017 12:35 232 353 720
I want:
Computer1 Fri 08/04/2017 13:20 110 917 340 907
Computer2 Fri 08/04/2017 13:21 110 917 435 852
Computer3 Fri 08/04/2017 12:39 180 92 916
Computer4 Fri 08/04/2017 12:35 232 353 720
I want to lead with a comma for every 3rd right-justified character, so the values line up correctly.
I am getting size of folders to calculate current folder size, then again weekly to determine growth.
The part I am struggling with is this:
for /f "tokens=1-2 delims= " %%a in ('C:\du64.EXE -v -q -nobanner C:\Temp^|find "Size:"') do SET DISKSIZE=%%b
ECHO. "%DISKSIZE%" **
(This will give a value containing commas. ex 12,345,678,910)
ECHO. %COMPUTERNAME%,%DATE%,%TIME:~0,5%,%DISKSIZE%,%PROCESSOR_ARCHITECTURE%>> "C:\DUOutput.CSV"
...set "DISKSIZE= %%b"
echo %disksize:~-15%
No idea why you're getting 92 in your data, nor what lead with a comma for every 3rd right-justified character means.
see set /?|more from the prompt for documentation. I've no idea how many spaces I put before %%b - so long as there is at least a string of 15, it should be OK.

Custom datetimeparsing to combine date and time after reading csv - Pandas

upon reading text file, I am presented with an odd format, where date and time are contained in separate columns, as follows (files is tabs as separators).
temp
room 1
Date Time simulation
Fri, 01/Jan 00:30 11.94
01:30 12
02:30 12.04
03:30 12.06
04:30 12.08
05:30 12.09
06:30 11.99
07:30 12.01
08:30 12.29
09:30 12.46
10:30 12.35
11:30 12.25
12:30 12.19
13:30 12.12
14:30 12.04
15:30 11.96
16:30 11.9
17:30 11.92
18:30 11.87
19:30 11.79
20:30 12
21:30 12.16
22:30 12.27
23:30 12.3
Sat, 02/Jan 00:30 12.25
01:30 12.19
02:30 12.14
03:30 12.11
etc.
I would like to:
parse date and time over two columns ([0],[1]);
shift all timestamps 30minutes early, that is replacing :30 with :00;
I have used the following code:
timeparse = lambda x: pd.datetime.strptime(x.replace(':30',':00'), '%H:%M')
df = pd.read_csv('Chart_1.txt',
sep='\t',
skiprows=1,
date_parser=timeparse,
parse_dates=['Time'],
header=1)
Which does seem to be parsing time not dates (obviously, as this is what I told it to do).
Also, skipping rows is useful for finding the Date and Time headers, but it discards the headers temp and room 1, that I need.
You can use:
import pandas as pd
df = pd.read_csv('Chart_1.txt', sep='\t')
#get temperature to variable tempfrom third column
temp = df.columns[2]
print (temp)
Dry resultant temperature (°C)
#get aps to variable aps from second row and third column
aps = df.iloc[1, 2]
print (aps)
AE4854c_Campshill_openings reduced_communal areas increased openings2.aps
#create mask from first column - all values contains / - dates
mask = df.iloc[:, 0].str.contains('/',na=False)
#shift all rows to right NOT contain dates
df1 = df[~mask].shift(1, axis=1)
#get rows with dates
df2 = df[mask]
#concat df1 and df2, sort unsorted indexes
df = pd.concat([df1, df2]).sort_index()
#create new column names by assign
#first 3 are custom, other are from first row and fourth to end columns
df.columns = ['date','time','no name'] + df.iloc[0, 3:].tolist()
#remove first 2 row
df = df[2:]
#fill NaN values in column date by forward filling
df.date = df.date.ffill()
#convert column to datetime
df.date = pd.to_datetime(df.date, format='%a, %d/%b')
#replace 30 minutes to 00
df.time = df.time.str.replace(':30', ':00')
print (df.head())
date time no name 3F_T09_SE_SW_Bed1 GF_office_S GF_office_W_tea \
2 1900-01-01 00:00 11.94 11.47 14.72 16.66
3 1900-01-01 01:00 12.00 11.63 14.83 16.69
4 1900-01-01 02:00 12.04 11.73 14.85 16.68
5 1900-01-01 03:00 12.06 11.80 14.83 16.65
6 1900-01-01 04:00 12.08 11.84 14.79 16.62
GF_Act.Room GF_Communal areas GF_Reception GF_Ent Lobby ... \
2 17.41 12.74 12.93 10.85 ...
3 17.45 12.74 13.14 11.00 ...
4 17.44 12.71 13.23 11.09 ...
5 17.41 12.68 13.27 11.16 ...
6 17.36 12.65 13.28 11.21 ...
2F_S01_SE_SW_Bedroom 2F_S01_SE_SW_Int Circ 2F_S01_SE_SW_Storage_int circ \
2 12.58 12.17 12.54
3 12.64 12.22 12.49
4 12.68 12.27 12.48
5 12.70 12.30 12.49
6 12.71 12.31 12.51
GF_G01_SE_SW_Bedroom GF_G01_SE_SW_Storage_Bed 3F_T09_SE_SW_Bathroom \
2 14.51 14.61 11.49
3 14.55 14.59 11.50
4 14.56 14.59 11.52
5 14.55 14.58 11.54
6 14.54 14.57 11.56
3F_T09_SE_SW_Circ 3F_T09_SE_SW_Storage_int circ GF_Lounge GF_Cafe
2 11.52 11.38 12.83 12.86
3 11.56 11.35 13.03 13.03
4 11.61 11.36 13.13 13.13
5 11.65 11.39 13.17 13.17
6 11.68 11.42 13.18 13.18
[5 rows x 31 columns]

How do I query with a join getting all the data in Om Next?

In Om Next, when having data such as:
{:table {:name "Disk Performance Table"
:data [:statistics :performance]}
:chart {:name "Combined Graph"
:data [:statistics :performance]}
:statistics {:performance {:cpu-usage [45 15 32 11 66 44]
:disk-activity [11 34 66 12 99 100]
:network-activity [55 87 20 1 22 82]}}}
you can query it with:
[{:chart [{:data [:cpu-usage]}]}]
to get the chart, join the data and dig down cpu-usage from the performance record:
{:chart {:data {:cpu-usage [45 15 32 11 66 44]}}}
How do I get the whole performance record instead?
Another potential query is this:
[{:chart [:data]}]
but it doesn't resolve the join:
{:chart {:data [:statistics :performance]}}
There are no components as this is only about the data and the query. This is from the exercise number 2 and queries here: https://awkay.github.io/om-tutorial/#!/om_tutorial.D_Queries which uses om/db->tree to run the queries.
This is how you do it:
[{:chart [{:data [*]}]}]
which gives you:
{:chart {:data {:cpu-usage [45 15 32 11 66 44]
:disk-activity [11 34 66 12 99 100]
:network-activity [55 87 20 1 22 82]}}}
Without seeing the actual components with queries and idents, I can't be sure.
However, you should be able to query for [{:chart [:data]}]. See om/db->tree. Assuming that you have structured your components with the right queries and idents, om/db->tree converts your flat app state into a tree so that your read functions see the following data when called:
{:table {:name "Disk Performance Table"
:data {:cpu-usage [45 15 32 11 66 44]
:disk-activity [11 34 66 12 99 100]
:network-activity [55 87 20 1 22 82]}}
:chart {:name "Combined Graph"
:data {:cpu-usage [45 15 32 11 66 44]
:disk-activity [11 34 66 12 99 100]
:network-activity [55 87 20 1 22 82]}}}
If that query doesn't work, [{:chart [{:data [:cpu-usage :disk-activity :network-activity]}]}] should certainly do the trick.