For example i would like to input a list of students into a database, can you suggest any formatted csv file or excel?.. what would be the best?..any suggestions would be appreciated! thanks! :D
There are far more libraries available for a wider range of languages to work with CSV than Excel files, so if those are the only two choices, I'd strongly suggest CSV.
But if you're putting them into a database, you could probably write a nice tool to either prompt a human to enter all the data into a database, or scrape an existing source of student information to push into the database. I mean, the whole point of having the database is so you can write queries, so might as well write some tools for putting data in too. :)
Related
I've been using Dax to help me Document my Power BI file. Using Dax queries I've been able to record all the fields that exist in the file, including calculated and measured fields. In my documentation process I am also looking to find a way to record visualizations on the report - namely the charts and graphs. Unfortunately, no Dax query I've read about provides a list of data such as the visualization title, what fields it's using, or what kind of graph it is. Is there any Dax query that provides this information, as a whole or any part of it?
In addition to attempting to document with Dax I have also looked at the raw XML data in the Power BI file (For those who may not know, you can rename your Power BI file from .pbix to .zip and view the raw data). The relevant files within PBI are either XML or JSON. Looking at ../Report/Layout.JSON specifically I have seen JSON-formatted text that includes visualization data. Is there any easy way to extract this data and format it in a more-readable fashion?
For clarity, I do not need the contents of the tables, but I would like a way to record what fields are being used in the visualization, rather than what fields merely exist.
EDIT: I've found a workaround. It isn't efficient, and I would still appreciate any knowledge on this subject
I mentioned going through the the Layout file, renaming it to .JSON and poking it in Notepad++. I've found that you can ctrl+f for "displayName", "queryRef" and ""title\":show\":true,\"text\":\"". Break these all to new lines and indent them with tab (Use ctrl+h and replace with \n\t in notepad). These indent the JSON-formatted lines for Power BI pages, fields called by visualizations, and the visualization titles (if they have any), respectively.
Save this document as .csv and load it into Excel by delimiting on tabs. Use your preferred process - I prefer query editor - editor to remove the other non-indented rows. There still may be a lot of excess characters on the indented lines which need to be removed manually. At the end of this process, though, I ended with 3 columns in excel listing the aforementioned fields I've been looking for.
On a PBIX file with more than a dozen pages and several hundred dependent fields this process took about three hours. If there are any faster ways to do this, I would love to hear about them
As you have noted, DAX doesn't help you in this case because it will tell you about the model rather than the visuals on the report pages. The Layout file works, but you have to parse it for the information you need. You could probably just pull that JSON file into Power BI and process it there to get the info you want. There are also third party tools that can help with this. I just looked at https://app.datavizioner.com/ and it lists the ID of the visual, the type of visual, and each field used in the visual. It is currently free and just requires you to upload a PBIT of your report. It doesn't have the title of the visual that we see, so you would have to find a way to map the IDs you see to the human-friendly title of the visuals if you need that.
See http://radacad.com/power-bi-helper. It can tell you tables and columns in use. It also can export a list of all tables, columns, formulas, and roles in your model.
If you want stuff on the visualizations and how they are configured, Layout.json is the only way I know. The file does open nicely in Power Query if you were so inclined to try to make something of it.
My new Power BI comparer tool documents the whole Power BI file (pbit). The "CompareVisuals"-tab should provide you with all the information necessary.
It also superfast: Just fill in the path to the pbits (you can fill in the same path into both fields, if you don't want to compare, but just to analyze one file).
https://www.thebiccountant.com/2019/09/14/compare-power-bi-files-with-power-bi-comparer-tool/
Is there any good way to migrate existing database from Domino Server to Relational database like MySQL without using any tool.
I've explored a bit about this and got to know that its possible using XML but don't know how and what'll be the procedure.
Any help would be appreciated.
Without using any tool: NO.
There are two big difficulties in exporting data:
First is the Notes Richtext, which is a proprietary format that has to be "transcoded" somehow. This is not an easy thing to do "manually" and needs either a lot of coding or some kind of tool.
Second is the fact, that there is no "forced" structure in Notes documents. There can be several forms that "define" how the documents look and there can be different versions of these forms that have been used over the past. A document may or may not contain any number of fields in any thinkable type (the field may even be number in one document and text in the other).
You have to KNOW the structure of your documents to get them out. Of course you can simply export them as "Structured Text" or as "Comma separated values", to get -most- of it, but then you need views that show the documents in the order you need them. Exporting them as XML is another "standard" way to get the data, but then you need to understand the xml to get it into your relational database.
Short: Without (at least very little) coding knowledge OR a tool (that costs money) there is no chance for getting the data out.
Ah yes, there is an "ODBC driver" for Lotus Notes / Domino, but that will not help you much, if you do not know the structure of your documents and how Notes- Databases work, it will also not work.
As Torsten said above, you can't do it without a tool, either you buy one or write one yourself.
I wrote a tool like that several years ago to export Notes databases as XML. There is a bit of work, especially with the rich text fields. You also may want to export/detach attachments and embedded images.
You can read more about my export tool here: http://www.texasswede.com/websites/texasswede.nsf/Page/Notes%20XML%20Exporter
I need to modify csv files via automated scripts. I need help in what direction I should look towards and what language the script should be in
Situation: I have a simple CSV file but I need an automated script that can edit certain fields and fill in blank ones with whatever I specify. What should my starting point be and what kind of a developer should I look for? Which coding language should he or she be knowledgable at?
Thank you!!
maybe you are looking for CSVfix, this is a tool for manipulating CSV data in the command shell. Take a look here: https://code.google.com/p/csvfix/
With it you can, among other things:
Reorder, remove, split and merge fields
Convert case, trim leading & trailing spaces
Search for specific content using regular expressions
Filter out duplicate data or data on exclusion lists
Enrich with data from other sources
Add sequence numbers and file source information
Split large CSV files into smaller files based on field contents
Perform arithmetic calculations on individual fields
Validate CSV data against a collection of validation rules
and convert between CSV and fixed format, XML, SQL and DSV
I hope this helps you out,
best regards,
Jürgen Jester
I am building a website with ROR 3. I need to provide a page to my clients wherein he could edit his pricing info regarding the application. I am quite confused on how to do this. The pricing page needs to be displayed as an html table with different columns which has got the pricing info.
I am thinking of different ways to do this.
1) Allow the client to create and upload an html page and then save it as a file in my public directory and render as an when the client clicks on the pricing link.
2) The clients may not have bare technical knowledge, hence make the client upload some other formats like Word, Excel etc and then parse it and store it as an HTML file in the public directory.
3) Provide the client with some real time editing tools where in the client could edit in a fixed format, and after wards save the file and render it later.
Also, I wouldn't like to store these infos in my database. There would be quite a few number of clients and hence managing all these data in my database would become cumbersome. Storing all these as plain html files and rendering it later would be the most ideal thing for me.
There might be other better steps in doing this as well. Could you please suggest which might the better, or any other option that could suit my needs? Basically I would want my clients to have a mechanism where they could provide there pricing details, edit it later and display it back as an html table, all this without using an Database backend. Any suggestions would be mostly appreciated.
Good way is Excel(csv format).
You can do PHP with Excel. I thing this is the best solutions for your requirement.
Try this.
http://php.net/manual/en/function.fgetcsv.php
If you are give authority to user to change edit contain and you have to used " CSV or Excel" please see these links:
Importing CSV and Excel
Exporting CSV and Excel
If you really don't want to use database then you can use YAML as a structured storage.
e.g. ( you, most probably, could come up with a better structure )
SMS_Pack:
Sl_No:
1: 10000
2: 25000
3: 50000
You can read those .yml files and parse them as hashes. Should be fairly easy to represent that hash as a HTML table.
For the creation, I'm sure you can come up with some dynamic form input. Or to just let the client send this kind of file ( which might not be the best solution ).
But it just might be easier to manage all of this information within a database.
I have a commercial ColdFusion application, running on a MySQL database. A possible new client has approached me, they have been working in a Lotus Notes environment (and their own database) for many years now. Ofcourse they want to migrate their data to my application, before making the move.
I'm trying to get a grip on how to get a thorough feeling of the data, structure and interdependencies in their current database-application. Are there any tools to see a database-structure (like in a RDBMS) of a NSF-file, or is there anyway to dump the structure using ColdFusion etc....I don't have any handson experience with Lotus Notes (I do in the meanwhile have a local Lotus client and their database).
I need a good startingpoint to be able to determine whether or not I can find a way to migrate the data.
Any ideas??
thanks
Bart
To get at the data in Notes, a good option is to use NotesSQL which can be found here:
A quick overview of the Notes data structure is this: Notes is a document-centric database, with non-relational data contained within each document. Notes Databases (NSFs) contain any number of Notes Documents, which in turn contain any number of items that hold data. Each Notes Document can have a different set of items, and thus different data in it. While that sounds like a horrible mess, usually the documents have similar data based on the form used to create the documents.
This all leads to why there is no simple way to get data out of Lotus Notes. There are a few other options, which may or may not be useful depending on how much data you have to migrate.
I personally like using XML to extract data from Lotus Notes. You can do so by creating XML views within a Notes database. IBM has a tutorial that looks helpful.
Using Java or LotusScript, you can write code to extract data from the documents to any format you wish (CSV, XML, TXT, etc)
If it's not a lot of data, you may find getting the data into an Excel format is the simplest intermediary step. Long ago I wrote an add-in tool for exporting data from Lotus Notes to Excel, which may help you. Or you can use the "Edit > Copy Selected To Table" feature in the Lotus Notes client to copy what is visible in a Notes View to the clipboard, and then paste that into Excel. In that scenario, you'd want to edit the views so they show all the data you need.
I hope this helps!