I'd like to use a CSV file as a lookup table to update some attributes.
So I figured the LookUpAttribute processor was what I needed. I configured it with as SimpleCsvFileLookupService as the Lookup Service, but I can't get it to work yet.
My SimpleCsvFileLookupService is configured but stays in a "enabling" state, and the LookUpAttribute processor still tells me it's "invalid because performing validation depends on referencing a Controller Service that is currently disabled".
I dont understand why it doesn't enable. Has somebody used these components ? Thx
Edit :
I didn't see the message in the left. It says the mapping for "1" is not found ("1" is set as the lookup key column and in the csv the header row is "1;2;3;4;5;6;7;8".
What am I missing ? I can't find any explanation as to how to use this controller service.
Edit2 : The SimpleCsvFileLookupService properties
Edit3 : Extract of the csv file
I want to bulk-load some emails in the Gsuite account using a csv file. However there's this field called "building id" which is NOT REQUIRED but is handy for my requirements; I just realized that out of the box, Gsuite cannot locate a record by the mentioned field as the search term. My question is, is there a workaround to this problem? Better yet, does Gsuite support the creation of custom fields that can be used to discover information when used as search strings? Anyone who knows a solution to this problem?
When using the Directory API to list the members, using the query parameter to get the members with a specific buildingId is not an option. The available queries that can be made are listed here.
However, you can create your own custom field using the below Directory API schemas.insert request:
POST https://admin.googleapis.com/admin/directory/v1/customer/{customerId}/schemas
With the following request body:
{
"schemaId": string,
"schemaName": string,
"fields": [
{
"displayName": "",
"fieldName": "",
"fieldType": ""
}
],
"displayName": string,
}
Afterwards, when you want to retrieve the users with a particular value, you will have to make the below request:
GET https://admin.googleapis.com/admin/directory/v1/users
With the following fields:
customFieldMask > set to the name of the schema;
projection > set to custom;
query=schemaName.FIELD='VALUE' > this will query the users who have the field from the custom field with the value 'VALUE'.
Reference
Directory API Users:list;
Directory API Schemas.insert.
Bigsql is not able to read data from subdirectory like hive can by setting parameters.
set tblproperties (
"hive.input.dir.recursive" = "TRUE",
"hive.mapred.supports.subdirectories" = "TRUE",
"hive.supports.subdirectories" = "TRUE",
"mapred.input.dir.recursive" = "TRUE")
I tried adding above parameters into bigsql tableproperties but it's not able to read subdirectory data.
What parameters I need to set in bigsql to read subdirectory data?
As per my understanding, Bigsql only looks up for files in parent directory. It doesn't even bother checking sub-directories. It will display an empty table because it cannot read the data recursively.This feature is still under product improvement ideas with IBM product and engineering team.
I send over proper json formatted code, according to v2 api docs:
lists/subscribe.json
"GROUPINGS":[{"id":removed_id,"name":"grouping_name","groups":["group_name"]}]
I get back information about a member... It does not say they are interested in that group name..
lists/member-info.json
"GROUPINGS":[{"id":removed_id,"name":"grouping_name","form_field":"hidden","groups":
[{"name":"group_name","interested":false},{"name":"other_group_name","interested":false},
{"name":"other_group_name2","interested":false},{"name":"other_group_name3","interested":false}]}]
I do not understand how I can get these users to show up as "subscribed" to a group within my grouping. I have been trying for nearly 5 hours now. I have tried:
making sure the groups: value is an [] array.
trying out making that value a string (which surprisingly did not throw errors)
capitalizing "GROUPINGS" when sending it in merge_vars
not capitalizing "groupings" when sending in merge_vars
using the lists/update-member.json method to update these groups
using the name of my interest group instead of grouping in the grouping array.
using the option replace_interests on both true and false
In conclusion,
I had to email mailchimp a support ticket. Without changing my code at all - it works this morning. Interestingly enough, mailchimp was experiencing many issues yesterday with servers being down and alleged email hackings.
To be clear about which version of my code worked - following the API exactly for v2.
It was an error on mailchimp's end.
{
"id":"MY LIST ID",
"email":{
"email":"THE EMAIL TO SUBSCRIBE"
},
"merge_vars":{
"GROUPINGS":[
{
"id":THE GROUPING ID,
"groups":[
"THE NAME OF MY GROUP"
]
}
]
},
"double_optin":false,
"replace_interests":true,
"apikey":"MY API KEY"
}
Also, in case you are curious, replace_interests is true when the groups you send are supposed to replace existing groups the subscriber is interested in. False indicates that the groups you pass should be added to the interest groups.
If you are updating a member (method: lists/update-member), it may be best to set replace_interests to false just to make sure it does not overwrite your existing interest groups (possibly with blank groups). The default value for it is true, so this could be another place prone to error.
In an SSIS project, I know how to set up a variable to hold the connection string for a Connection Manager - but how do I go the other way? ie - I have an existing SSIS package and want to find out the name of the variable which provides the connection string for a particular Connection Manager.
I can find loads of references to how to set it up, and did expect that going to Connection manager>Properties>Expressions would show me, but it doesn't. I did manage to find it by going to Package Configuration Organizer, picking sensibly named parameters, going to edit them, going to the second dialog of the wizard to find the Exported property name. This can't be the only way, surely?
Regards, Stewart
EDIT - This is in Visual Studio 2008
First of all, grab BIDSHelper it's a free add-in for visual studio and at a minimum it helps identify when elements have Expressions and Configurations applied to them. One will have a teal highlight, the other a fuschia colored and yes, an object can have both.
The first scenario you described, look at the Properties, Expressions and identify the use of expressions. Other objects, you might need to look at an Expressions tab.
Configurations work differently. You can use an Environment Variable, Registry Value, Parent Package Value, XML file or SQL Server table. The first three provide a 1:1 mapping between a configuration value and a configured item (variable, connection manager, etc). XML and SQL Server can configure many items. The order configurations are applied is important as you could have 5 configuration entries and each one of them modify the same setting with different values. There is also a difference between how 2005 and 2008 applies configurations so take a peek at Understanding How Integration Services Applies Configurations
When a package loads, BIDS will indicate what configurations it is attempting to load (look in your output window). Beyond BIDS highlighting and those messages, those messages are your other clue that configurations exist and are being applied. That's also your opportunity for detecting missing configurations (I expected to find configuration X and didn't find it = the configuration resource doesn't exist) or (I expected to configure property X but could not find it = the thing being configured does not exist)
I have found the best approach is to define a common set of configurations (Sales connection, warehouse connection) that all the applications in an environment use and use a consistent configuration naming approach. We then use custom configurations for project level things (the path for input and output for the InsuranceProcessing packages is would apply across all the packages but would be different for Sales) and then a third set of configurations that is package specific. We use SQL Server tables for this as it makes inspecting values much easier than gloming through lots of ugly XML.
Lots of information, but nothing that directly answers your question. Sorry about that. You might be able to inspect the object model and look at what's configured but that's ugly.
Thanks to the above answers, I realised that it wasn't a variable, but a configuration. Once that became clear, more Googling led to an explanation of why the Target Object is blank in the Package Configuration Organizer (there can be more than one).
The answer is in the config database; the PackagePath entry holds \Package.Connections[connection manager name].Properties[ConnectionString], so to find out where a particular mapping comes from, use something like this.
SELECT TOP 1000 [ConfigurationFilter]
,[ConfiguredValue]
,[PackagePath]
,[ConfiguredValueType]
FROM [Database].[dbo].[Configurations_table]
where ConfiguredValue like '%object in which you are interested%'
(not sure why all that didn't go in the code block).
I still think it should be easier, but I hope this helps others.
Regards, Stewart
Great answer by billinkc. In addition to that answer (or rather fleshing out the "inspect the object model and look at what's configured" part), I run a script at the start of any package that adds the values of all connection manager connection strings to the output window, followed by the connection string expression for each manager. In addition it loops through all the variables that have been specifed for use in the script and outputs the value. Not so useful in production but very useful when developing/testing.
Just add a script task to the start of the package flow, specify any variables you want to debug then add the following code to the script:
'Report number of connections
Dts.Events.FireInformation(99, "debug", "number of connections = " & Dts.Connections.Count, "", 0, True)
'Loop through connection collection
For Each cConnection As Microsoft.SqlServer.Dts.Runtime.ConnectionManager In Dts.Connections
'Report connection string value
Try
Dts.Events.FireInformation(99, "debug", "connection """ & cConnection.Name & """ value = " & cConnection.ConnectionString, "", 0, True)
Catch
End Try
'Report connection string expression
Try
Dts.Events.FireInformation(99, "debug", "connection """ & cConnection.Name & """ constring expression = " & cConnection.GetExpression("ConnectionString"), "", 0, True)
Catch
End Try
Next
'Report number of variables
Dts.Events.FireInformation(99, "debug", "Number of Variables = " & Dts.Variables.Count, "", 0, True)
'Loop through variables collection
For Each vVariable As Microsoft.SqlServer.Dts.Runtime.Variable In Dts.Variables
'Report variable value
Try
Dts.Events.FireInformation(99, "debug", "Variable """ & vVariable.Name & """ value = " & vVariable.Value, "", 0, True)
Catch
End Try
Next