How many rows of data can karate "read(file.csv)" can read? - json

I am trying to read and print the data from a csv file using below karate scenario outline code snippet. the CSV file contains more than 350 rows of data with 2 columns "Description" and "Expression". But when I execute the below code snippet its running for more than 1hr and not generating any cucumber reports.
Note: the same code snippet works fine when the CSV file contains 50-100 rows
Scenario Outline: <Description>
And def result = {"items":{"name":'#(Description)',"expression":'#(Expression)'}}
Then def rule_description = result.items.name
Then print rule_description
Then def rule_expression = result.items.expression
Then print rule_expression
Examples:
| data_file |
Can someone please help and guide me what am I missing here.

Related

Importing specific columns from a CSV into excel

I am trying to do what the title says and also do it for new records. I cannot link the CSV file because it exceeds the 255 limit. So i am attempting to split up the table.
I have the below table in access
DateOfTest
Time
PromptTime
TestSequence
PATResults
Logs
Serial Number
1
2
3
4
5
6
7
Obviously, where the numbers are i want the data from the CSV to be inserted.
I have created a form including a button so i can run some VBA, but i cannot find the correct information online for my work, as i am new to VBA it is also a bit confusing.
I have attempted some random code, but i was just spraying and praying at that point
I am not sure I understood your question. In the impoer tool you can choose columns, but if you want to do it with a script, I would suggest to perform pre-processing phase with simple python and pandas to read the csv file, remove any unwanted columns and save to another CSV to be uploaded directly to excel.
something like this
import pandas as pd
df = pd.read_csv ('csvfile.csv')
df.drop('column_name', inplace=True, axis=1)
df.to_excel ('filename.xlsx', index = False, header=True)

Sending Redshift Query Result in Email with Python and Airflow

I am making a daily DAG that will run a query in Redshift and will send the result table in an email.
I currently manage to take the query results and put it into a DataFrame. My problem is using send_email (from airflow package) and choosing the parameters for it to show the data.
The output of the email is unfortunately not a table, just the results being all over the place as text.
Here is what I wrote for send_email :
html_content = f"""
<html><body><p>Hello,</p>
<p>Found {num_of_late_dags} Late DAGs:</p>
{late_dags}
<p>Regards,</p>
<p>Me</p>
</body></html>
"""
send_email(to=v_email_recipients_daily_report,
cc=cc,
bcc=bcc,
subject=f'Daily DAG Load Report',
html_content=html_content,
mime_subtype='alternative') ```
Update:
I used pd.to_html() after html_content in .format(to_html()) but that didnt work.
I used to_html() when creating the data as a DataFrame it worked. Not sure why, they should do the same.
So it looked like this: df = pd.DataFrame(data, columns=['X',Y']).to_html()
Then putting df inside the HTML content worked.

How do you import a json file to excel with tree-references?

I have a .json file that has around 8 Megabyte of data, containing multiple HashSets from Powershell with the Command:
$Hashset = #{"ChildHashset" = #{"ChildOfChildHashset" = #{...} ,... }, ...}
$Hashset | Convertto-Json | out-file "C:\...\file.json"
I want to work with the Json-File in Excel, so I Need the Parent-Level to be in a seperate Table, as the Child-Level.
As I tried to Import the .json file, The Values of the Hashset-Names were [Record] without a reference.
Additionally I had to manually click on every record, Excel has shown, to generate a Table from it.
Is there a Method to simple Input the .Json-File, and generate multiple Excel-Tables with designated Name & Reference to previous Data-Value?
Thanks in advance.

Analysis of deviance table model output in HTML

I am trying to export the output of an 'Analysis of deviance table' in HTML format, so that it can be inserted into a word document.
I created a GLM model as follows:
newmod <- glm(cbind(Recaptured, predated) ~ Morph * Plant * Site, data =
survival, family = binomial)
Running the following code gives me the output that I would like to export to HTML:
anova(newmod,test="Chisq")
I have tried the following code to create a HTML table using stargazer, however it doesn't seem to be working:
anova_mod<-anova(newmod,test="Chisq")
stargazer(newmod, type="html", out = "anova_output.htm")
Is there a simple way of doing this in r? I have managed to successfully export the summary statistics, but what I really need is the Analysis of deviance table.
I believe you are looking for:
print(xtable(anova_mod), type = "html")
as indicated by this answer: Exporting R tables to HTML
Here is my full code for reproducing something similar to your question:
plant.df = PlantGrowth
plant.df$group = factor(plant.df$group,labels = c("Control", "Treatment 1", "Treatment 2"))
newmod = lm(weight ~ group, data = plant.df)
anova_mod=anova(newmod)
anova_mod
install.packages("xtable")
require(xtable)
print(xtable(anova_mod), type = "html")
You can then paste the output to an html vizualizer such as: https://htmledit.squarefree.com/ to see the resulting table.
Instead of printing it, you can write it to a file. I have not personally tested this part, but the second answer in this question should work for you: Save html result to a txt or html file
Note: You can also reference all parts of the anova_mod separately by adding a $ after it like anova_mod$Df.

Converting a JSON file of unknown structure to CSV using R

I am looking for a way to convert a rather big (3GB) json file to csv. I tried using R and this is the code that I used.
library("rjson")
data <- fromJSON(file="C:/POI data 30 Rows.json")
json_data <- as.data.frame(data)
write.csv(json_data, file='C:/POI data 30 Rows Exported.csv')
The example I am using is only a subset of the total data of about 30 rows. which I extracted using EMeditor and copied and pasted into a text file. The problem is however it only converts the first row of the data.
I am not an experienced programmer and have tried everything on youtube tutorials from php to excel and nothing seems to work. The problem is I have no Idea what the structure of the data is so I can not create a predetermined data frame and there is a number of missing values within the data.
Any advice would be greatly appreciated.