Hi guys I'm currently trying to create a database for a business which works with unit sensor data. A company has multiple monitoring units which has multiple sensors, each sensor spits out CSV files every hour or so.
Im trying to relate this data to a company, so clients can view their sensors when they enter the site. Im currently just importing csv scripts into custom tables (by custom i mean just custom headers), obviously the table can only be as big as the CSV table (needs to be the same amount of rows). But the issue i am having is trying to link the csv data to the company? The CSV's don't have id numbers so i cant use keys to link tables.
Here is my Schema: https://puu.sh/wRnPs/eff7a60b5b.png
Here is a CSV example: https://puu.sh/wRoPA/09b3146d40.png
Any ideas would be greatly appreciated i have done a lot of research and can't seem to find a solution!
Related
On our Wordpress site, we use a plugin called s2member and it stores the levels (roles) of our clients as well as the times they were assigned a specific level in our database. I would like to create a table that shows when a user was assigned a specific level. I'm having a challenge getting the data I need because of the way the data is stored in the field. It stores all of the levels along with the associated dates and times when a user's level was changed in one field. In addition, it stores all of the times as Unix timestamps. Here's an example of a typical field associated with a client:
a:20:{s:15:"1562695223.0001";s:6:"level0";s:15:"1562695223.0002";s:6:"level1";s:15:"1562695223.0003";s:6:"level2";s:15:"1562695223.0004";s:6:"level3";s:15:"1577906312.0001";s:11:"ccap_prepay";s:15:"1596575898.0001";s:12:"-ccap_prepay";s:15:"1596575898.0002";s:13:"ccap_graduate";s:15:"1596575898.0003";s:11:"ccap_prepay";s:15:"1596575898.0004";s:7:"-level3";s:15:"1597196952.0001";s:14:"-ccap_graduate";s:15:"1597196952.0002";s:12:"-ccap_prepay";s:15:"1597196952.0003";s:13:"ccap_graduate";s:15:"1597196952.0004";s:11:"ccap_prepay";s:15:"1598382433.0001";s:14:"-ccap_graduate";s:15:"1598382433.0002";s:12:"-ccap_prepay";s:15:"1598382433.0003";s:11:"ccap_prepay";s:15:"1598382433.0004";s:6:"level3";s:15:"1605290551.0001";s:12:"-ccap_prepay";s:15:"1605290551.0002";s:11:"ccap_prepay";s:15:"1605290551.0003";s:13:"ccap_graduate";}
There are four columns in this table: umeta_id; user_id; meta_key; meta_value. The data above is stored in the meta_value column.
You'll notice that it also has multiple ccap_* entries. CCAP stands for custom capapability and I would like to be able to chart those assignments and associated times as well.
Do you have any idea how I can accomplish this?
Thank you for any help you can give.
I talked to an engineer about this and he told me that I would need to learn Python and I believe he said I would need to learn how to also use Pandas and Numpy to extract the data I need but he wasn't exactly sure. I started taking a data analyst course on Coursera but I still haven't learned what I need to learn and it's already been several months. It would be great if someone could provide a solution that I could implement more quickly and use on an ongoing basis.
If there's a way to accomplish my goal by exporting this table to a CSV file and using Microsoft Excel or Google Sheets, I'm open to that too.
Here's an image of the table (if it helps):
Database table
Here's an example of my desired output:
Desired output
In my desired output, I used Excel and created a column that converts the Unix timestamp to a short date and another column where I used a nested IF statement to convert the CCAP or level to its meaning that we understand internally.
Good afternoon everybody,
I hope you are all doing well.
I have an excel file with two tabs:
Data
Workings
On the data one, I have tables from a website https://www.soccerstats.com/team.asp?league=spain&stats=10-real-madrid where you can see the teams stats.
The problem is that for, only a few teams, some of the tables are not working. For instance, for Real Madrid, the website table named “Scoring Patterns” works properly when I export (linked the data) to Excel by using a query. However, for other teams like Real Sociedad when going to Data on the ribbon, then get & transform from web and then pasting the URL https://www.soccerstats.com/team.asp?league=spain&stats=14-real-sociedad when I find the table “Scoring Patterns” it has another name. When I select the table that have another name but contains the same data as for the “Scoring Patterns” for Real Madrid, it gives me an error. It says that it couldn´t find the Goal/No Goal column. Why is it happening?
Thanks
I am building a web application that will run off of data that is produced for the public by a governmental agency. The issue is that the csv file that houses the data I need is a 2,000 column beast of a file. The file is what it is, I need to find the best way to take it and modify it. I know I need to break this data up into much smaller tables within MySQL, but I'm struggling with the best way to do this. I need to make this as easy as possible to replicate for next year when the data file is produced again (and every year after). I've searched for programs to help, and everything I've seen deals with a huge amount of rows, not columns. Has anyone else dealt with this problem before? Any ideas? I've spent the last week color coding columns in excel and moving data to new tabs, but this is time consuming, will be super difficult to replicate and I worry it leaves me open for copy and paste errors. I'm at a complete loss here!
Thank you in advance!
I suggest that you use functions in excel to give every column an automatic name "column1", "column2", "column3", etc.
After that import the entire csv file into MySQL.
Decide on which columns you want to group together into separate tables. This is the longest step and no program can help you manage this part.
Query your massive SQL table to get just the columns you want for each group. Export these queries to CSV and then import them as new tables in your database.
At the end, if you want, query all the columns you didn't put into separate groups. Make this a new table in the database and delete the original table to save on storage space.
Does this government csv file get updated and republished in the same format every time? If so you'll need to write a script to do all of the above automatically.
I’ve searched but can’t find exactly what I’m trying to do, or just losing my mind... probably both.
I’ve got a data feed, with the following fields: Product ID, Name, Retail Price, Sales Price. Right now I dump this into a Google Sheets file via scripts, trim the necessary fields, export via CSV and back in MySQL but I need the data to go right from the feed CSV URL to MySQL, and correct a few fields.
For some reason, some of the prices have weird values, such as 4728.7376 while others have 282. I’d like to basically trim andything past the decimal point. When I tried that, I have some rows that don’t have any prices (it’s annoying) and seems to break the import.
Any suggestions on simplifying this to pull from the URL feed, fix issues and import into MySQL — which is through Google Cloud, but currently testing using MAMP. I can only get it to work importing using varchar(100) and need to do calculations with those columns; as long as the value is present.
Thanks in advance!
im a bit of a newbie in access and i hope im not asking a stupid question. I have recently had to move an inventory system from excel to access. Each product is recipted in tbl.rct and has an order number a lot number quantity and expiry.
Each individual lot number needs to be verified before it can be recipted this information is on tbl.lot.
While making a form to receipt products i noticed that i couldnt add any products without their lot number already on lot.tbl - is there a way to get around this?
http://imgur.com/kCc7G39
Attached relationships
I think you mix between Excel and the Access. These Table imported directly from the excel without any requirement change to meet Access Goals. The Database use to reduce the repeat routine work. The Tables that most be (Products, Order, Receipts, Lot must be Stock and collect data of (Qty, lot#, expiry, damage). Now we make sequence to how insert to Database. Open New Receipt to include in the stock the Product(link ID) and the detail. This is now in the warehouse. For selling you will make invoice when select product will show you the Lot available and its expiry and of course you select filter to filter on FIFP LIFO.
You can send me the excel file to convert to database if yes please provide me more information because the flow not clear well