My team keeps sending the output files of regressions and we extract them into excel. Specifically the model summary and the Coefficients tables.
Is there a way that I can get OMS to extract the tables from existing output files? Or will I have to run the regressions again?
OMS works only from the Statistics backend at the time the output is generated. You can't use it on output. However, you can use OUTPUT EXPORT to create Excel files from Viewer objects. The problem with this in some scenarios is that OUTPUT EXPORT run from syntax doesn't give you control over which tables are exported, although you can select them manually in the Viewer and use OUTPUT EXPORT to export all those selected.
If that is an issue for you, you can use the SPSSINC MODIFY OUTPUT extension command with a custom function included with the command to select the tables to export based on the OMS table types. Here are some examples.
Export a custom table from the most recent command to a new Excel file.
SPSSINC MODIFY OUTPUT TABLES
/IF SUBTYPE="'Custom Table'" PROCESS=PRECEDING
/CUSTOM FUNCTION="customoutputfunctions.excelexport(file='c:/temp/extest.xls')".
Export all the custom tables in the Viewer to separate sheets named table1, table2, ...
SPSSINC MODIFY OUTPUT TABLES
/IF SUBTYPE="'Custom Table'" PROCESS=ALL
/CUSTOM FUNCTION="customoutputfunctions.excelexport(file='c:/temp/extest.xls',
sheet='table#',action='CreateWorksheet')".
Export all custom tables to separate files named extest1,xls, extest2, ...
SPSSINC MODIFY OUTPUT TABLES
/IF SUBTYPE="'Custom Table'" PROCESS=ALL
/CUSTOM FUNCTION="customoutputfunctions.excelexport(file='c:/temp/extest#.xls')
You can install this command from the Utilities menu in Statistics 22-23 or the Extensions menu in V24. The installation includes the customoutputfunctions.py module, but if you have an old version of SPSSINC MODIFY OUTPUT, you might need to update it. The excelexport function was added in 2/2013.
HTH
Related
Might be going about this completely the wrong way - happy to be shown the error of my ways.
In a nutshell, I've got 50-odd files of mixed types (csv and excel) that I want to import (each file to its own table) to an SQL database.
In the control flow I've got an sql task that returns:
The source data filename
The source data filetype (csv / xlsx)
What I want to name the table to import to.
This object gets passed to a Foreach loop that loops through this object and puts these 3 fields into variables.
I want to then say "if the filetype variable is csv, go and do a flat file import. If it's .xlsx, go and do an excel import"
So inside my for each container I've got a dataflow task.
I want the first thing the dataflow task does to check the filetype variable, and then do the appropriate import.
I think it's got to be in the dataflow, because there isn't an "If" style control I can see in the control flow?
But I'm at a loss as to how I pass a variable into the conditional split.
Any thoughts welcome.
OR! - just had a thought. Is the best way to do this to get a list of all the csv file types, process them in a dataflow, then get a list of all the .xlsx ones and process them - so I'd have:
Get csv filenames & tablenames
for each to loop through these
dataflow to import data from csv
get xlsx filenames and tablenames
for each through these
dataflow to import data from xlsx.
Just doesn't seem as elegant?
Cheers
I am trying to create a derived column based on columns provided in different input file but unfortunately I keep getting error when I tried to map my Raw_File_1 with Derived Column.The error looks like this:
Cannot create a connector.
The destination component does not have any available inputs for use in creating a path.
My goal is to able to connect both Raw_File_1 and Map_File_1 into Derived Column and generate a new column.
If anyone can provide me any suggestion that would be great!!
I have source file and reference file both are flat file. My source file has column a, column b and column c and my reference file has column d, column e and column f.
If column a=column d and column b=column f then I want to populate column c as the same value as column f. How can I do this kind of analysis or lookup in SSIS
Based on your comments that I patched into the question, you're looking to augment the existing data based on matching data from your reference file.
The core of your SSIS package will look like this
In the first data flow, we will source from map_file_1 and load into a "raw" file.
I configure my raw file destination like this
When the package runs, it'll fill that special format file with the reference data. It's important, because you can either use a database or a raw file as your lookup source.
Finally, we get to work! A flat file source to a Lookup component. In the first tab of that lookup, be sure to change the Connection type from the default of "OLE DB connection manager" to "Cache connection manager"
In Connection tab, click to create a new CCM and use the raw file generated in the preceding step.
Map columns A to D and B to E (assuming data types match). Click the check box on column F and in the Lookup Operation part, Replace C with that value.
Final thoughts
This will be a case sensitive lookup. If things don't have a match in the reference file, it's going to blow up. That's probably not what you want so configure the Lookup transformation to not do that ;)
I blogged about using Excel to populate the cache if you want more words http://billfellows.blogspot.com/2011/11/using-excel-in-ssis-lookup.html
Your question is not clear, i will try to give some suggestions:
If you are looking to perform a lookup with a derived column:
You can use Cache Transform component and Cache connection manager to achieve that:
SSIS - How To Use Flat File Or Excel File In Lookup Transformation [Cache Transformation]
If you are looking to Merge both input:
Then you need to use Merge Join or Union All components:
SSIS Union All Transformation
Learn SSIS : MERGE, MERGE JOIN and UNION ALL
SSIS Basics: Using the Merge Join Transformation
I am new to Talend os.
However, I received a task:
Create file delimited .csv metadata (one for Lead & Opportunity).
Move files to your repository on the AWS server (the etl_process1 login).
Create two tables sfdc_leads_reporting_raw and sfdc_opp_reporting_raw.
Load the data from the files into the tables. Ensure the data types are correctly used when creating metadata schemas & tables.
Till step 4 I am done.
Now the problem is:
How to Implement logging at the end of each job to report the number of leads (count of distinct id in leads table) and number of opportunities created (count of opportunity id) by stages (how many converted, qualified, closed won, and dead)?
Help would be appreciated.
You can get this data using global variables, in a subjob at the end of your job. Most components provide a global variable called tComponent_NB_LINE (or _NB_LINE_INSERTED for database components) that gives you the number of lines output by the component.
For instance tFileOutputDelimited_1_NB_LINE or tOracleOutput_1_NB_LINE_INSERTED.
Using these variables you can log into console or file.
Here is a simple example. If you have a tOracleOutput_1 in your job you can do:
tPostJob -- OnComponentOk -- tFixedFlowInput -- Main -- tLogRow
Inside tFixedFlowInput you retrieve the variable
(Integer)globalMap.get("tOracleOutput_1_NB_LINE_INSERTED")`.
If you need to log aggregated info, you can append a tAggregateRow to your output components, and use tSetGlobalVar to get count by certain criteria.
I have a report where I have added a data driven subscription that writes the report to a folder location on a daily schedule. The report is rendered as csv. the file name comes as MyReport1234.csv.
I need this to be Myreport1234.txt. How can I do this in SSRS?
I use SSRS 2008 R2 btw
found a solution for this:
Create a data driven subcscription
on Step 3, just write SELECT 1 or if you're trying to generate a dynamic file you could embed the logic for this in your select statement and give it an alias like SELECT 'Myreport1234.txt' as [FileName]
on Step 4, either Specify a static file name, or get the value from the database using the column "FileName" which you created in the previous step.
set the Render Format to CSV
This is the important step >> File Extension must be set to False. This will stop the renderer from adding the default extension for the renderer of your choice. leaving whatever thats on the filename to appear as the file extension.
SSIS
This is general question which i am going to ask and might possible to have multiple answer.
Techie, please share with me the best solution if familiar with scenario.
i have 100 customer and they share file for us to load into our database.
and after compution and the output file need to be saved in Output folder.
the name of the output file should be customer_name.txt where customer_name
is coming from a column available in input file.
Can anyone please help me how should i design this in SSIS to achieve my Goal
At first, use a Foreach Loop Container for every file.
For each one, store data into your importTable in database with the fileName or customerName.
Then, when you need to get output file:
Get the nameFile or userName with a query and save it into a ssis variable.
Create your file with the 'DestinationVariable' as 'User::yourVariableName'
Add the data from your database and then use a simple script to move it into output folder
Example