I open a table inside the GUI of MS Access. I mark a cell which contains:
Myriam
I hit ctrl + c to copy it. I open a new document in Notepad++ and copy with strg + v. The result is:
"Myriam
Myriam"
I get two lines instead of one! There are 27 thousand entries in that column and ONLY for this one I observe this behaviour. I was able to track it down to this level, but now I'm clueless about the 'why' ... ?
Using the "Zoom" feature (ShiftF2) in Access revealed that the field actually did contain
Myriam
Myriam
This was probably due to a user accidentally hitting CtrlEnter during data entry, which added the newline and made the field look empty, so they typed the name in a second time.
Related
Disclaimer: new to SSIS and Active Directory
I have a need to extract all users within a particular Active Directory (AD) domain and import them into Excel. I have followed this: https://www.itnota.com/query-ldap-in-visual-studio-ssis/ in order to create my SSIS package. My SQL is:
LDAP://DC=JOHN,DC=JANE,DC=DOE;(&(objectCategory=person)(objectClass=user)(name=a*));Name,sAMAccountName
As you know there is a 1,000 row limit when pulling from the AD. In my SQL I currently have (name=a*) to test the process and it works. I need to know how to setup a loop with variables to pull all records and import into Excel (or whatever you experts recommend). Also, how do I know what the other field names are that are available to pull?
Thanks in advance.
How do I see what's in Active Directory
Tool recommendations are off topic for the site but a tool that you can download, no install required, is AD Explorer It's a MS tool that allows you to view your domain. Highly recommend people that need to see what's in AD use something like this as it shows you your basic structure.
What's my domain controller?
Start -> Command Prompt
Type set | find /i "userdnsdomain" and look for USERDNSDOMAIN and put that value in the connect dialog and I save it because I don't want to enter this every time.
Search/Find and then look yourself up. Here I'm going to find my account by using my sAMAccountName
The search results show only one user but there could have been multiples since I did a contains relationship.
Double clicking the value in the bottom results section causes the under pane window to update with the details of the search result.
This is nice because while the right side shows all the properties associated to my account, it's also updated the left pane to navigate to the CN. In my case it's CN=Users but again, it could be something else in your specific environment.
You might discover an interesting categorization for your particular domain. At a very large client, I discovered that my target users were all under a CN
(Canonical Name, I think) so I could use that in my AD query.
There are things you'll see here that you sure would like to bring into a data flow but you won't be able to. Like the memberOf that's a complex type and there's no equivalent in the data flow data types for it. I think Integer8 is also something that didn't work.
Loop the loop
The "trick" here is that we'll need to take advantage of the
The name of the AD provider has changed since I last looked at this. In VS 2017, I see the OLE DB Provider name as "OLE DB Provider for Microsoft Directory Service"
Put in your query and you should get results back. Let that happen so the metadata is set.
An ADO.NET source does not support parameterization as the OLE DB does. However, you can apply an Expression on the Data Flow which surfaces the component and that's what we'll do.
Click out of the Data Flow and back into the Control Flow and right click on the Data Flow and select Properties. In that properties window, find Expressions and click the ellipses ... Up pops the Property Expressions Editor
Find the ADO.NET source under Property and in the Expressions section, click the Ellipses.
Here, we'll use your same source query just to prove we're doing the right things
"LDAP://DC=JOHN,DC=JANE,DC=DOE;(&(objectCategory=person)(objectClass=user)(name=" + "a" + "*));Name,sAMAccountName"
We're doing string building here so the problem we're left to solve is how we can substitute something for the "a" in the above query.
The laziest route would be to
Create an SSIS variable of type String called CurrentLetter and initialize it to a
Update the expression we just created to be "LDAP://DC=JOHN,DC=JANE,DC=DOE;(&(objectCategory=person)(objectClass=user)(name=" + #[USer::CurrentLetter] + "*));Name,sAMAccountName"
Add a Foreach Loop Container (FELC) to your Control Flow.
Configure the FELC with an enumerator of "Foreach Item Enumerator"
Click the Columns...
Click Add (this results in Column 0 with data type String) so click OK
Fill the collection with each letter of the alphabet
In the Variable Mappings tab, assign Variable User::CurrentLetter to Index 0
Click OK
Old blog posts on the matter because I like clicks
https://billfellows.blogspot.com/2011/04/active-directory-ssis-data-source.html
http://billfellows.blogspot.com/2013/11/biml-active-directory-ssis-data-source.html
I have 350 files of data with each containing about 4,000 rows. There are 3,000 unique rows but some rows are duplicated e.g.
"2021-02-02",20.1,99,0,3.4
"2021-02-03",22.6,95,0,2.9
"2021-02-04",18.8,90,0,5.2
"2021-02-02",20.1,99,0,3.4
"2021-02-03",22.6,95,0,2.9
"2021-02-05",21.9,96,0.8,4.2
"2021-02-06",20.8,95,0,3.3
I will like to remove only the duplicate lines in each of the 350 files. However, the duplicate lines are different in each file. i.e., some files may have other dates duplicated apart from the sample shown. The duplicate lines are random and not in any particular order. I used Line Operations in Notepad++ to sort the lines in ascending order and then remove duplicates. It works okay for one file but it will take a long time repeating this step 350 times.
As mentioned in comments a script in your favorite scripting language is the best way.
But you may have a look at the screenshots below and try for your needs.
I assume you have all files or part of them in one directory. Please think about a backup copy for your test.
Open one file in your workspace
Open the dialog e.g. by STRG+F
Try for your needs Find What: ^(.*?)$\s+?^(?=.*^\1$)
Choose Regular Expression and matches newline
Open Find in Files tab e.g. by STRG+Shift+F
Replace with: Nothing
Set Filter
Set Directory
Press Replace in Files (at your own risk!)
Before:
After:
I have an access table of vendors, used by many queries and vba. The end users populate the fields (in a form) based on W9 data in Excel, which often contains line breaks (aka character turns, line feeds). If they paste a piece of data which contains line break, then it breaks many other processes. I need to find a way to prevent them pasting data with line breaks, and, prevents them from manually entering a line break into the field. I have tried various vba options to find and remove but this can still result in bad data.
Example: they paste in a vendor number that has a line break from excel such as:
1234
1246
The various find/replace functions can convert this to 1234.1246, or take the first set before the line break: 1234, or the second set after the line break: 1246. But there is no way for the database to know which 1234 or 1246 is correct.
I need a way to 1) prevent the data from being input and 2) to notify the end user that the data they attempted to input was not accepted.
Please help,
Sincerely,
Pulling My Hair Out in Chunks
I figured it out with the help of a coworker!!
Validation Rule:
Not Like "*[ insert line break here
]*"
This won't even let them save if they force close the database!!
I have a txt file available on the web which contains tab separated values (TSV/CSV) like this:
Product_IdtabColortabPricetabQuantityItem1 tabRed tab$5.2 tab5Item2 tabBlue tab$7.5 tab10
I imported the txt file into a Google Spreadsheet using the IMPORTDATA(url) formula. The problem is that now I need to split the text to columns. I tried the following formulas without success:
Split(A1,"\t")
Split(A1," ")
Split(A1,"<tab>")
another thing I tried is to to use the Substitute function, but I just can't figure out how to match the Tab character in Google Spreadsheets?
Pages strips tabs by default when you paste text using a standard paste. Tab delimited data can be pasted and automatically parsed using:
Right Click -> Paste special -> Paste values only
IMPORTDATA(url) seems to handle tabs automatically, as others have mentioned before, if the URL ends in ".tsv".
I had trouble trying to import a file from Dropbox even though the file was named "something.tsv", because the url was
"https://www.dropbox.com/s/xxxxxxx/something.tsv?dl=1"
I managed to solve the problem by adding a dummy query parameter to the url:
"https://www.dropbox.com/s/xxxxxxx/something.tsv?dl=1&x=.tsv"
NOTE: I know this question was asked back in 2014 and I am answering this question some 5 years later. I am posting the answer here in hopes that someone else who googles their way here will be saved the headache and can be helped by how I devised a solution.
SUMMARY OF THE ISSUE: By default the IMPORTDATA() function will properly process a tab-delimited file only if the file name ends with the extension .TSV
UPDATE Nov 14, 2019:
In a comment below, Poul shared that he has found an undocumented parameter for the IMPORTDATA() function by which you can specify the delimiter to split the data. As of writing this, the official documentation makes no reference to this delimiter.
In effect the documentation should look something like the following:
IMPORTDATA("url","delimiter")
So, if you wanted to force a file to be split on the TAB character, it would look something like
IMPORTDATA("url","\t")
PRIOR ANSWER:
UPDATE: I am leaving my original answer just in case it might be helpful if the answer above, which includes undocumented functionality, does not continue to work.
ORIGINAL ANSWER: After seemingly countless attempts, I figured out how to coax Google Sheets into importing a tab-delimited file regardless of the extension.
For those looking for the quick and dirty answer, copy the following into a cell of a Google Sheet to give it a try:
=ARRAYFORMULA(IFERROR(SPLIT(IMPORTDATA("https://iso639-3.sil.org/sites/iso639-3/files/downloads/iso-639-3_Latin1.tab"),CHAR(9),FALSE,FALSE)))
For those that want to know a bit more, I will try to explain how each of the nested functions are helping to create the final solution:
=ARRAYFORMULA( IFERROR( SPLIT( IMPORTDATA(URL-HERE) ,CHAR(9),FALSE,FALSE) ) )
IMPORTDATA() - the primary function that pulls in the data file from the web
SPLIT - split the row by tab, note the use of char(09) to generate the tab character; also note the use of FALSE for the last parameter which was required in my case to ensure empty cells were not collapsed together
IFERROR - used to catch situations where an import might fail, the error will be trapped and not returned to the spreadsheet
ARRAYFORMULA - this function ensures that every line in the file is parsed; without this, only the first line of the file would be returned to the spreadsheet
It turns out that IMPORTDATA(url) can import a tab separated file, but it expects the file name to have the .tsv extension. This is inconsistent with Excel, where a tab-separated export results in *.txt.
If you can ensure that you use a .tsv extension, then your problem is solved.
You can also use the Sheets UI to import the file (into a new Spreadsheet). Select File > Import..., then Upload > Select a file from your computer. When the file selection dialog opens, paste the URL into the file name field, and click Open. The file will be downloaded to your PC then uploaded to Drive, through the Import dialog that will let you choose the delimiter.
(Validated on Windows 8.1 with Chrome; I don't know how this will behave on other OSes or browsers.)
Edit: See this gist.
importFromCSV(string fileName, string sheetName)
Populates a sheet with contents read from a CSV file located in the user's GDrive. If either parameter is not provided, the function will open inputBoxes to obtain them interactively.
Automatically detects tab or comma delimited input.
I had luck using split() and indicating only a single space as the delimiter, even though the data i pasted in had tabs separating each "column": =SPLIT(A1, " ", True) where A1 had data separated by 1 or more spaces. It seems that pasting in TSV data results in conversion from tabs to spaces.
This could be done in two steps leveraging the fact that tab is essentially multiple spaces.
Steps are as follows:
Select the columns which have tab separated data. Then trim tab to single space by using Data -> Data cleanup -> Trim whitespaces.
Now usual Data -> Split text to columns should work out of the box or after selecting space as separator.
In Vim 7, Ctrl-X Ctrl-O shows a list of possible values but I find this sequence of keys to be too long when I frequently use the autocomplete feature. For instance, in an HTML file, I'd like to see the list automatically popup after I type a < followed by one or two letters. In a CSS file, I'd like to see the list after I hit the ":" key. Is there a way to set this up?
To activate the omnicompletion on typing a ":" you can use
the following mapping.
imap : :<c-x><c-o>
The disadvantage is that each time you press ":" omnicompletion will
be activated, even when typing ":" in comments or in any other context
in which you do not want omnicompletion.
I have mapped ctrl-space to active omnicompletion:
imap <c-space> <c-x><c-o>
This gives me the choice to activate omni whenever I need it.
Another alternative that I found easier is just to press tab two times when you want autocomplete, and one time for regular tab.
Add the following line to your ~/.vimrc
imap <tab><tab> <c-x><c-o>