Disclaimer: new to SSIS and Active Directory
I have a need to extract all users within a particular Active Directory (AD) domain and import them into Excel. I have followed this: https://www.itnota.com/query-ldap-in-visual-studio-ssis/ in order to create my SSIS package. My SQL is:
LDAP://DC=JOHN,DC=JANE,DC=DOE;(&(objectCategory=person)(objectClass=user)(name=a*));Name,sAMAccountName
As you know there is a 1,000 row limit when pulling from the AD. In my SQL I currently have (name=a*) to test the process and it works. I need to know how to setup a loop with variables to pull all records and import into Excel (or whatever you experts recommend). Also, how do I know what the other field names are that are available to pull?
Thanks in advance.
How do I see what's in Active Directory
Tool recommendations are off topic for the site but a tool that you can download, no install required, is AD Explorer It's a MS tool that allows you to view your domain. Highly recommend people that need to see what's in AD use something like this as it shows you your basic structure.
What's my domain controller?
Start -> Command Prompt
Type set | find /i "userdnsdomain" and look for USERDNSDOMAIN and put that value in the connect dialog and I save it because I don't want to enter this every time.
Search/Find and then look yourself up. Here I'm going to find my account by using my sAMAccountName
The search results show only one user but there could have been multiples since I did a contains relationship.
Double clicking the value in the bottom results section causes the under pane window to update with the details of the search result.
This is nice because while the right side shows all the properties associated to my account, it's also updated the left pane to navigate to the CN. In my case it's CN=Users but again, it could be something else in your specific environment.
You might discover an interesting categorization for your particular domain. At a very large client, I discovered that my target users were all under a CN
(Canonical Name, I think) so I could use that in my AD query.
There are things you'll see here that you sure would like to bring into a data flow but you won't be able to. Like the memberOf that's a complex type and there's no equivalent in the data flow data types for it. I think Integer8 is also something that didn't work.
Loop the loop
The "trick" here is that we'll need to take advantage of the
The name of the AD provider has changed since I last looked at this. In VS 2017, I see the OLE DB Provider name as "OLE DB Provider for Microsoft Directory Service"
Put in your query and you should get results back. Let that happen so the metadata is set.
An ADO.NET source does not support parameterization as the OLE DB does. However, you can apply an Expression on the Data Flow which surfaces the component and that's what we'll do.
Click out of the Data Flow and back into the Control Flow and right click on the Data Flow and select Properties. In that properties window, find Expressions and click the ellipses ... Up pops the Property Expressions Editor
Find the ADO.NET source under Property and in the Expressions section, click the Ellipses.
Here, we'll use your same source query just to prove we're doing the right things
"LDAP://DC=JOHN,DC=JANE,DC=DOE;(&(objectCategory=person)(objectClass=user)(name=" + "a" + "*));Name,sAMAccountName"
We're doing string building here so the problem we're left to solve is how we can substitute something for the "a" in the above query.
The laziest route would be to
Create an SSIS variable of type String called CurrentLetter and initialize it to a
Update the expression we just created to be "LDAP://DC=JOHN,DC=JANE,DC=DOE;(&(objectCategory=person)(objectClass=user)(name=" + #[USer::CurrentLetter] + "*));Name,sAMAccountName"
Add a Foreach Loop Container (FELC) to your Control Flow.
Configure the FELC with an enumerator of "Foreach Item Enumerator"
Click the Columns...
Click Add (this results in Column 0 with data type String) so click OK
Fill the collection with each letter of the alphabet
In the Variable Mappings tab, assign Variable User::CurrentLetter to Index 0
Click OK
Old blog posts on the matter because I like clicks
https://billfellows.blogspot.com/2011/04/active-directory-ssis-data-source.html
http://billfellows.blogspot.com/2013/11/biml-active-directory-ssis-data-source.html
Steps
1.Created a user database - DB1
2.Created a table - TB1 with columns - cm1 and cm2
3.Installed the extension - "Database Administration Tool Extension"
4.Relaunched the application
5.Right clicked on the database - DB1 ................................
Observation -> I could see some options like 'Manage' but not properties
6.Right clicked on the column 'cm1' under the created table TB1.........................
Observation -> I could see only refresh option but not properties
7.Additional Step: Tried searching for other relevant extension but couldn't spot.
Question -> How can I get the properties option enabled for DB and columns in AzureDataStudio in MAC System
When performing long-running queries on Aurora Serverless I have seen the following errors after a few minutes:
ERROR 1080 (08S01) Forcing close of thread
ERROR 2013 (HY000) Lost connection to MySQL server during query
The queries are using mysql LOAD DATA LOCAL INFILE to load large (multi-GB) data files into the database.
How can I avoid these errors?
To solve this, you can change the parameter group item net_write_timeout to a more suitable value. Here's instructions for completing the steps from the console:
Go to RDS Console
Click "Parameter Groups" in the left pane
Click "Create Parameter Group"
On the Parameter Group Details page, for Type, choose DB Cluster Parameter Group; then give it a name and description, and click "Create"
Click on the name of the parameter group you created in step 4
Search for "net_write_timeout"
Click on the checkbox next to the parameter and click "Edit Parameters"
Change the value to an integer between 1-31536000 for the number of seconds you want it to wait before timing out, and click "save changes"
Click on Databases in the left pane
Click on the database and click "modify"
Under Additional Configuration > Database Options > DB Cluster Parameter Group, select the parameter group you created in step 4, and click "Continue"
Select "Apply Immediately" and click "Modify Cluster"
Break up your large, multi-GB uploads into smaller chunks. Aurora works better (and faster) loading one hundred 10MB files at once rather than one 1GB file. Assuming your data is already in a loadable format:
Split the file into parts using split
split -n l/100 --additional-suffix="_small" big_file.txt
This results in 100 files like xaa_small xab_small, etc.
find files that match the split suffixes using find
files=$(find . -name 'x*_small')
loop through the files and load each in parallel
for file in $files; do
echo "load data local infile '$file' into table my_table;" |
mysql --defaults-file=/home/ubuntu/.my.cnf --defaults-group-suffix=test &
done
I am using SQuirrel SQL Client Version 3.5 and everytime I modify and run a query it opens up a new result tab. I would prefer that it re-use the existing result tab (replacing the contents) Is there a way to do this?
Another way to do this (without having to set the sticky flag everytime) is to set the maximum limit of SQL Result Tabs to 1 in the Global New Session Properties.
You'll find it here: File --> New Session Properties --> SQL Tab
Right click on a result tab and select sticky. This prevents new result tabs from opening.
I am getting an error message: "Operation must use an updateable query" when I try to run my SQL. From my understanding, this happens when joins are used in update/delete queries in MS Access. However, I'm a little confused because I have another query almost identical in my database which works fine.
This is my troublesome query:
UPDATE [GS] INNER JOIN [Views] ON
([Views].Hostname = [GS].Hostname)
AND ([GS].APPID = [Views].APPID)
SET
[GS].APPID = [Views].APPID,
[GS].[Name] = [Views].[Name],
[GS].Hostname = [Views].Hostname,
[GS].[Date] = [Views].[Date],
[GS].[Unit] = [Views].[Unit],
[GS].[Owner] = [Views].[Owner];
As I said before, I am confused because I have another query similar to this, which runs perfectly. This is that query:
UPDATE [Views] INNER JOIN [GS] ON
[Views].APPID = [GS].APPID
SET
[GS].APPID = [Views].APPID,
[GS].[Name] = [Views].[Name],
[GS].[Criticial?] = [Views].[Criticial?],
[GS].[Unit] = [Views].[Unit],
[GS].[Owner] = [Views].[Owner];
What is wrong with my first query? Why does the second query work when the first doesn't?
There is no error in the code, but the error is thrown due to the following:
- Please check whether you have given Read-write permission to MS-Access database file.
- The Database file where it is stored (say in Folder1) is read-only..?
suppose you are stored the database (MS-Access file) in read only folder, while running your application the connection is not force-fully opened. Hence change the file permission / its containing folder permission like in C:\Program files all most all c drive files been set read-only so changing this permission solves this Problem.
Whether this answer is universally true or not, I don't know, but I solved this by altering my query slightly.
Rather than joining a select query to a table and processing it, I changed the select query to create a temporary table. I then used that temporary table to the real table and it all worked perfectly.
I had the same error when was trying to update linked table.
The issue was that linked table had no PRIMARY KEY.
After adding primary key constraint on database side and re linking this table to access problem was solved.
Hope it will help somebody.
I had the same problem exactly, and I can't remember how I fond this solution but simply adding DISTINCTROW solved the problem.
In your code it will look like this:
UPDATE DISTINCTROW [GS] INNER JOIN [Views] ON <- the only change is here
([Views].Hostname = [GS].Hostname)
AND ([GS].APPID = [Views].APPID)
...
I'm not sure why this works, but for me, it did exactly what I needed.
To update records, you need to write changes to .mdb file on disk. If your web/shared application can't write to disk, you can't update existing or add new records. So, enable read/write access in database folder or move database to other folder where your application has write permission....for more detail please check:
http://www.beansoftware.com/ASP.NET-FAQ/Operation-Must-Use-An-Updateable-Query.aspx
set permission on application directory solve this issue with me
To set this permission, right click on the App_Data folder (or whichever other folder you have put the file in) and select Properties. Look for the Security tab. If you can't see it, you need to go to My Computer, then click Tools and choose Folder Options.... then click the View tab. Scroll to the bottom and uncheck "Use simple file sharing (recommended)". Back to the Security tab, you need to add the relevant account to the Group or User Names box. Click Add.... then click Advanced, then Find Now. The appropriate account should be listed. Double click it to add it to the Group or User Names box, then check the Modify option in the permissions. That's it. You are done.
I used a temp table and finally got this to work. Here is the logic that is used once you create the temp table:
UPDATE your_table, temp
SET your_table.value = temp.value
WHERE your_table.id = temp.id
I got this same error and using a primary key did not make a difference. The issue was that the table is a linked Excel table. I know there are settings to change this but my IT department has locked this so we cant change it. Instead, I created a make table from the linked table and used that instead in my Update Query and it worked. Note, any queries in your query that are also linked to the same Excel linked table will cause the same error so you will need to change these as well so they are not directly linked to the Excel linked table. HTH
This is a shot in the dark but try putting the two operands for the AND in parentheses
On ((A = B) And (C = D))
I was accessing the database using UNC path and occasionally this exception was thrown. When I replaced the computer name with IP address, the problem was suddenly resolved.
You have to remove the IMEX=1 if you want to update. ;)
"IMEX=1; tells the driver to always read "intermixed" (numbers, dates, strings etc) data columns as text. Note that this option might affect excel sheet write access negative."
https://www.connectionstrings.com/excel/
UPDATE [GS] INNER JOIN [Views] ON
([Views].Hostname = [GS].Hostname)
AND ([GS].APPID = [Views].APPID) <------------ This is the difference
SET
[GS].APPID = [Views].APPID,
[GS].[Name] = [Views].[Name],
[GS].Hostname = [Views].Hostname,
[GS].[Date] = [Views].[Date],
[GS].[Unit] = [Views].[Unit],
[GS].[Owner] = [Views].[Owner];