I frequently need to pull some CSV reports and analyze them using powerpivot. The "issue" is that the tool spits out the report like this:
Report Name Keywords (Group contains 778600, Campaign contains us-en)
Client XYZ
Scope Entire Account
Date Range 3/12/2015
Filters Campaign contains us-en; Group contains 778600; Clicks > 0; Reduced Dimension
Keyword Account Publisher Campaign Group Search Bid $ Status Destination URL
Total for all 2 keywords
Keyword Account Publisher Campaign Group Search Bid $ Status Destination URL
bla bla bla Account Name Publisher Name Campaign Name Group Name 1 Active URL
So what i always need to do is to remove the first 9 rows of the CSV prior to importing. Usually i can do this on Notepad++, but sometimes the CSV is so large that i actually can't really open it to edit. So far i'm using a program called 010 Editor, but i have only some days left of it.
Is there an easy way to skip those rows when importing?
Thanks a lot
You can use Power Query (free to download) to load data to Power Pivot. It allows you to skip the first x rows and filter out rows with blank/null values. Once you are able to get this to work once, you can copy the M code to use it on other CSVs. Or you can automate it as a function and just feed it file locations.
Related
I am new to Jmeter and doing a POC to do a load test on a web application.
What I am trying to do:
I have a total of 4 user logins(surgeons). Each Login is associated with 'n' number of patients.
I've created 2 CSV files
one with the user login and password for surgeons
another CSV file that contains the PatientName, PatientID and the Surgeon associated with that Patient like below.
PatientName,PatientId,loginName
Pa1,PID1,user1
Pa2,PID2,user1
Pa3,PID3,user1
Pa4,PID4,user1
Pa5,PID5,user2
Pa6,PID6,user2
Pa7,PID7,user3
Pa8,PID8,user4
My Scenario:
Login as User.
Navigate to Each Patient Dashboard as per their associations.
log out of the application.
My Testplan
Thread Group (4 users, Ramp up time as 1 sec, 1 loop) -csv1(with username, password )
-Login Page and Navigate to the Main page - RunTime Controller (To sustain the load of a set amount of time)
-- While Loop(to loop between the patient dashboard of the surgeon/user logged in) ---CSV2 (the data as shown above) ----Navigate to Dashboard
----Navigate to Main
-Log out of the Application
What I want to achieve:
I want to use the single thread group and run it concurrently for all the 4 users. In this process, once the user login, the user should only those patient data from the CSV which are associated.
For Ex: When the Thread1 is running with User1 login, he should only able to loop through Pa1, Pa2, Pa3, Pa4 users When the thread2 is running with User2 login, user should only read the Pa5, Pa6 data.
Like this, each user login should only pick those users as per their associations mentioned above.
Is there any way, I can use this single CSV2 file and achieve this task? so that I don't have to create n number of the thread of n numbers of logins with n number CSV files each containing the data specific to the user login.
I did try to use the _CSVread function but that will make me to create multiple files(I currently have 500 CSV files) which is not a great idea. Expecting to find a solution to have all the data in one CSV and read it based on the Column value.
Reading data from CSV file based on particular column value is not supported in JMeter, you can consider the following options:
Create separate CSV files for each surgeon and pick up the relevant file based on currently logged surgeon id/name/whatever using __CSVRead() function.
Use If Controller to choose to this or that execution branch based on the surgeon name
Use Switch Controller to choose this or that execution branch based on the surgeon name
Objective: Use an API to feed information into a excel file.
I have an .csv file containing over 8k entries. I will run API requests in a row for those 8k entries and would like to save parts of the reply in another .csv file for all entries.
For example:
file1.csv contains Groups IDS( group1, group2, group3, ...).
Every API request using the parameter of Group ID would return a complete response with several entries like the one below:
<entry>
<id>https://example/api/v1/OData/GroupMemberships(GroupId='Group1',MemberId='iYa4qIi86asvbG')</id>
<title type="text">GroupMemberships(GroupId='Group1',MemberId='iYa4qIi86asvbG')</title>
<updated>2022-06-03T13:20:57+00:00</updated>
<author>
<name/>
</author>
<content type="application/xml">
<m:properties>
<d:GroupId m:type="Edm.String">Group1</d:GroupId>
<d:MemberId m:type="Edm.String">iYa4qIi86asvbG</d:MemberId>
<d:MemberType m:type="Edm.String">admin</d:MemberType>
...
...
</m:properties>
...
So in every iteration of the runner, I would like to extract only the information about MemberId and MemberType, and add those to an excel file that will contain all the information. I expect more than 8000 groups and +100k memberids
Every request for one groupID returns several entries containing all memberIDs for this group > save those specific entries to CSV > run next request and repeat.
This CSV file would look like this:
Group ID
Member ID
Member Type
Group1
iYa4qIi86asvbG
Admin
Group1
memberid2
User
Group1
memberId3
User
Group2
memberid1
User
Group2
memberid2
Admin
...
...
...
I am trying to use Postman to run a batch of API requests (using Collection Runner) in order to get the information I need and feed those specific parts of the response into columns of the CSV. I was researching how to achieve this, but the results were not quite what I was expecting (either they wanted me to write the response into Postman variables or they wanted me to use Newman, which looked like would be too difficult for my current knowledge of API requests)
Is there an easy way to perform this with Postman (maybe writing a script in the Pre-request or test?) or should I try a different approach?
I have a Google Sheet that contains extracted metadata from a large amount of files that I transferred from CDs to a server. I am currently working on creating a description for these materials to include in a finding aid. I found it easiest to work in Excel or Sheets because the PUI we use to output our finding aids utilizes a spreadsheet upload plugin.
I've been using pivot tables in Google Sheets to sort through all of the data with little issue except when I need to generate a date range for the files contained in one CD. Essentially, I'm creating a pivot table that contains rows for the URI, Filename (in this case I'm filtering for folder name only), and date_modified. The data looks something like this:
URI
FILENAME
DATE_MODIFIED
URI1
FOLDER1
2000-01-01
URI1
FOLDER2
2000-01-01
URI1
FOLDER3
2000-02-01
URI1
FOLDER4
1999-12-02
URI2
FOLDER1
2001-01-20
... and so on.
I'd like to generate a date range for each unique URI. I realize I could just go through each unique URI and manually extract that data but I have a LOT of these to go through so I don't think it is the most efficient use of my time. Especially, when you notice that the dates do not follow a chronological order. I'm thinking that pivot tables are not going to help me here so if anyone has other suggestions I'm happy to listen but brownie points if anyone has a solution that works in Sheets!
Try this on a new tab somewhere.
=QUERY(Sheet1!A:C,"select A,MIN(C),MAX(C) where A<>'' group by A")
change the range ref to suit.
Then in the next column over, depending on where you output the query,
=IF(A2="",,TEXT(B2,"yyyy-mm-dd")&"-"&TEXT(C2,"yyyy-mm-dd"))
drag down to the bottom.
To comply to regulations, I'm trying to download the purchase invoice documents (as PDF files) from some of my divisions to save them on-disk for archiving purposes.
I use Invantive Query Tool to do this. I like to know which table to use and how to export these attachments only regarding purchase invoice documents.
You can indeed do this by using the export options in Invantive Query Tool or Invantive Data Hub.
What you need is a query that hooks up the document information of type 20 (purchase invoices) with the actual attachment files. You can find a list of types and their description in the DocumentTypes view. You can find the document attachment files in the DocumentAttachmentFiles table.
When you have retrieved that, you can export the documents from that query to disk using a local export documents statement.
The full query is here:
use 123456
select /*+ join_set(dae, document, 10000) */ attachmentfromurl
, dct.division || '/' || dae.id || '-' || filename
filepath
from exactonlinerest..documents dct
join DocumentAttachmentFiles dae
on dae.division = dct.division
and dae.document = dct.id
where dct.Type = 20
order
by dct.division
, dae.id
local export documents in attachmentfromurl to "c:\temp\docs" filename column Filepath
Make sure to set the ID of the division right in the use statement (this is the technical ID, not the 'division number', which can contain duplicates). You can find that in the top menu bar under Partitions. Or simply use use all to get the documents from all divisions (this might take a while).
Also set the file path right where it says c:\temp\docs now. Then hit F5 in the Query Tool to execute, or run the script from Data Hub.
In Google Spreadsheets. I have a large sheet containing lots of bookings, a booking has several columns for name, voucher number, email address etc. Each date I have a class on covers several columns and rows. I want to build a single vertical list of every email address in the sheet. The email addresses are in different columns and different rows. I was thinking FILTER or LOOKUP but I can't quite find the answer that works.
Dec 17th 2-5
Name Voucher Email People
Richa yes richa#xx.ca 2
Jessica yes jess#sss.com 3
Beth cash ebeth#hh.com 2
Export your sheet as a CSV file (File > Download as ..)
Then use the REGEX tool of your choice (Notepad++) or some of the online tools available (first hit: http://emailregex.com/email-extractor-tool/).
You could use flatten() to get all the data in a single column and use regexextract(Cell or range,"[a-zA-Z]+#[a-zA-Z]+\.[a-zA-Z]+")