I have a complete shapefile, but I just need 30% of the .dbf records. I clean the .dbf easily in SQL, but the connection with the shapefile is lost.
So, how can clean the entire shapefile of the records I don't need?
Thanks!
OK I found the solution.
1) Make a query removing the un-needed elements: RightClick (RC) on Layer>Properties>Defintion Query> Query Builder > ... > ok
2) RC on Layer > Open Attribute Table > table Options Button > Select All Records > Close the Table
3) RC on Layer > Data > Export Data > Select target folder > Save as type "Shape file" in the scroll-down menu
4) Will ask if you want the new shape file added as layer in current project Y/N
5) Done!
More details here:
Related
I have two files that are named like this:
CustomerReport(08022021-08032021)
ComparingReport(08022021-08032021)
I need to load the CustomerReport to a table and the ComparingReport to another table.
I tried for each loop container but I cant think of how the expression will be to pickup the file.
Im thinking of something like Customer*.csv where the * acts like a wild card but that didnt work. What can I do in this case?
Here key of the answer is to use Foreach Loop and Conditional Split.
Don't pay attention on errors, because I don't have your CSV files and tables in DB!
Create a new variable FileName - string data type
Add foreach loop and set like on screenshot
Collection Tab:
Variable Mappings Tab:
Add Data Flow Task into Foreach Loop container
Drag elements from toolbox like on a image
Flat File Source connect to one of your CSV file using Flat File Connection Manager and on connection manager, in properties > expressions, set for ConnectionString a variable FileName
Set Conditional Split like on a image
Expression for Customer is:
LEFT(
SUBSTRING(#[User::FileName],FINDSTRING((#[User::FileName]),"Customer",1),100),
FINDSTRING(SUBSTRING(#[User::FileName],FINDSTRING((#[User::FileName]),"Customer",1),100),"Report",1) - 1
) == "Customer"
Connect Conditional Split to OLE DB Destination's
NOTE: I can't run package as I said on top of this answer, Pay attention to Conditional Split, but this is the way how you need to find if part of a string is into whole string.
I have about a hundred JSON files of data that I would like to be able to manipulate in Excel. The reason why there are so many files is that the API I pulled from limits responses to 50 items per request, so I chained 100 requests together in Postman and each request generated its own file.
The layout of each file is as follows:
{
"href": "dsjdsjds.com",
"total": 4293,
"next": "sdsadsads.com",
"prev": "dsjdjsdj.com",
"limit": 50,
"offset": 50,
"itemSummaries": [...]
}
Pretty much all of the data that I want lies inside the itemSummaries class.
I'm pretty new to this and not sure if the optimal way would be to use a Python script, or if there was a way to use VBA or something. I was thinking that I'd need to combine all of the data into a single file first, but I don't know how to do that either. I appreciate the help!
Here is what I did for similar situation where I had to import multiple JSON files, all with same structure.
Use Get & Transform in Data Ribbon to import the JSON file as text.
The Power Query will recognize this as JSON. Edit the result in Power
Query window and expand/transform the imported data until you can
show in tabular form.
You can then convert these manual sequence of
steps into Custom Function. See here for details -
https://www.poweredsolutions.co/2019/02/19/parameters-and-functions-in-power-bi-power-query-custom-functions/
Go back to Excel and this time instead of importing the JSON file, import the folder where all these JSON are available and apply your custom function on the individual JSON files to produce a consolidated table.
I found this article.
Import from JSON
Select Data > Get Data > From File > From Folder. The Browse dialog box appears.
Locate the folder containing the files you want to combine.
Note The message “No items match your search” means Power Query has found a folder and it’s displayed in the Folder name box. The files you want to combine are still in the folder, but just not visible.
A list of the files in the folder appears in the dialog box. Verify that all the files you want are listed.
Select one of the commands at the bottom of the dialog box, for example Combine > Combine & Transform. There are additional commands discussed in the section About all those commands.
The Power Query Editor appears.
The Value column is a structured List column. Select the Expand icon, and then select Expand to New rows.
The Value column is now a structured Record column. Select the Expand icon. A drop-down dialog box appears.
Keep all the columns selected. You may want to clear the Use original column name as a prefix check box. Select OK.
Select all the columns that contain data values. Select Home, the arrow next to Remove Columns, and then select Remove Other Columns.
Select Home > Close & Load.
Result
Power Query automatically creates a query to consolidate the data from each file into a worksheet. The query steps and columns created depend on which command you choose. For more information, see the section, About all those queries.
To comply to regulations, I'm trying to download the purchase invoice documents (as PDF files) from some of my divisions to save them on-disk for archiving purposes.
I use Invantive Query Tool to do this. I like to know which table to use and how to export these attachments only regarding purchase invoice documents.
You can indeed do this by using the export options in Invantive Query Tool or Invantive Data Hub.
What you need is a query that hooks up the document information of type 20 (purchase invoices) with the actual attachment files. You can find a list of types and their description in the DocumentTypes view. You can find the document attachment files in the DocumentAttachmentFiles table.
When you have retrieved that, you can export the documents from that query to disk using a local export documents statement.
The full query is here:
use 123456
select /*+ join_set(dae, document, 10000) */ attachmentfromurl
, dct.division || '/' || dae.id || '-' || filename
filepath
from exactonlinerest..documents dct
join DocumentAttachmentFiles dae
on dae.division = dct.division
and dae.document = dct.id
where dct.Type = 20
order
by dct.division
, dae.id
local export documents in attachmentfromurl to "c:\temp\docs" filename column Filepath
Make sure to set the ID of the division right in the use statement (this is the technical ID, not the 'division number', which can contain duplicates). You can find that in the top menu bar under Partitions. Or simply use use all to get the documents from all divisions (this might take a while).
Also set the file path right where it says c:\temp\docs now. Then hit F5 in the Query Tool to execute, or run the script from Data Hub.
Ive been searching the internet to what I thought would be a straight forward question to answer. Hope you guys can help?
I am using for each loop to look for specific files and move them with file system task to a different folder.
Say I have 10 csv files. called listed A to J
I only want to move a,e and j but cant seem to get the foreachloop to look for that group.
In the enumerator Files text box i have tried inserting the 3 file names split by various separators, but SSIS thinks its all one specific file and none of the 3 get moved.
Can someone advise how it can be done? Just to confirm, I dont to use wild card logic just group of specific file names - similur to the IN function of SQL query
Thanks in advance
now added img - please advise how to slect 3 specific files in the text box with arrow
Since OP isn't able to proceed with just my comment, I'll explain a bit more in detail -
Use an EXECUTE SQL TASK to dump the names of the files needed into an SSIS object (to do this you could use a stored procedure or an SQL query). Create an object-type variable in the variables tab prior, change the output in the EXECUTE SQL TASK to Full Result Set and map the result to the object you just created. Now this object holds the list of files you need to loop through.
Now drag-and-drop a ForEach container from the SSIS toolbox. It should be configured as a ForEach ADO Enumerator and map the object to it. Create another variable of type string that will hold the file names after each iteration of the ForEach container. Map this also in the Variables tab of the ForEach container.
Now, place the File System Task which you would use to move these files into the ForEach loop. Use the file-name-variable you created to move just the required files.
Now if you're not sure what SQL query to use for your case in step 1 to get the 3 file names -
SELECT 'A.csv'
UNION
SELECT 'E.csv'
UNION
SELECT 'J.csv'
I'm working on a Ssis package and would like to know, how I can achieve the following:
I want to move files from a drop folder into a process folder and I want to implement the following rule:
If file does not exist in archive move file to process and archive.
If file exists in archive drop file (don't archive and don't move to process).
The test "if" exists must be based on file name and time stamp (when raw file got created).
Any ideas?
You can do this by simple way which I have done this in few days back.
1) Create a variable FileName(string), FileExists(boolean)
2) Drag the File System Task and based on your condition you can Copy/Move/Delete file or folder.
3) In my case based on the time frame I archive the file which is move the file from one folder to another by adding one more variable name DestinationFolder (string).
4) The condition I applied is in Precedence Constraint (right click on properties or double click Precedence constraint editor then expression and constraint and give the expression as #fileexists == TRUE or FALSE).
This should work just fine.