I have an access table of vendors, used by many queries and vba. The end users populate the fields (in a form) based on W9 data in Excel, which often contains line breaks (aka character turns, line feeds). If they paste a piece of data which contains line break, then it breaks many other processes. I need to find a way to prevent them pasting data with line breaks, and, prevents them from manually entering a line break into the field. I have tried various vba options to find and remove but this can still result in bad data.
Example: they paste in a vendor number that has a line break from excel such as:
1234
1246
The various find/replace functions can convert this to 1234.1246, or take the first set before the line break: 1234, or the second set after the line break: 1246. But there is no way for the database to know which 1234 or 1246 is correct.
I need a way to 1) prevent the data from being input and 2) to notify the end user that the data they attempted to input was not accepted.
Please help,
Sincerely,
Pulling My Hair Out in Chunks
I figured it out with the help of a coworker!!
Validation Rule:
Not Like "*[ insert line break here
]*"
This won't even let them save if they force close the database!!
Related
I am currently working on an input file and I do have a column which contains 3 different values in one cell itself. Although this data is not being used in the transformation , I need to input this data from the source and then ignore when it is loaded into the staging table.
But the issue I face is that it gets loaded into separate rows rather than 1 cell.
This particular column is input as a string datatype. what change do I need to make to resolve this issue. Please let me know If more details are needed to answer the question.
I have uploaded a sample file to google drive https://drive.google.com/file/d/17hn8xmRd4CWsgKBzHgdwnR9W4jTJ9lTn/view?usp=sharing
The following is a screenshot of the csv data as opened in a text editor
Having downloaded sample.csv from your link, the first thing I did was open it in a text editor (Notepad++, TextPad, Visual Studio, etc) and just looked at what you have.
Row 1 is column headers
Encoded in UFT-8 with BOM (byte order marker)
Line Endings are CR/LF (Carriage Return & Line Feed)
Column delimiter appears to be a comma ,
Double Quote, ", is used as the text qualifier but only when needed
There are CR/LF characters in the actual data
I then define my flat file connection manager based on that data
Finally, I have a data flow with a Flat File Source to a Derived Column and drop a Data Viewer between them
As you can see, configuring your Flat File Connection Manager as I show will allow all the data to flow into your table as expected.
What is happening now is the CRLF, which is our row delimiter, is having precedence over the embedded CRLF in the column data. By setting the double quote as the Text Qualifier, the data reader correctly "skips" the embedded CRLF until it is encountered outside of the quotes.
Using Apache Hop latest version I'm trying to read in a plain text file. This text file is old and basically only structured by its lines (it has no delimiter, no seperator, no enclosure, etc.). I would like to read and process the lines of this file as rows in my transformation.
I use the "Text file input" transformation to read the file. Apparently reading it works, but I seem have no field available when trying to retrieve the fields. It simply states that no fields were found.
When I run the "preview records" I do get empty records equal to the number if lines in the file, so that is good. However there is no data shown as there is no field detected.
Curiously enough, when I press "Show file content" I DO get the desired content, nicely structured in the rows as desired, so I know the file is being read correctly.
Does anyone know how to best read these kind of files?
PS: The files can be anywhere from 10 to 100000 lines.
When there is no header row with field names or Hop is not able to detect any fields you can also create a field in the fields tab and it will put content in there.
As we just use a position based approach and split the content using the specified delimiter everything should go in "field1" when no delimiter is found in the data.
Figured it out. The naming is a bit misleading, but you can use the "CSV File input" and then set a TAB as delimited. Then use preview on your file and you should find that the lines are actually being parsed.
I open a table inside the GUI of MS Access. I mark a cell which contains:
Myriam
I hit ctrl + c to copy it. I open a new document in Notepad++ and copy with strg + v. The result is:
"Myriam
Myriam"
I get two lines instead of one! There are 27 thousand entries in that column and ONLY for this one I observe this behaviour. I was able to track it down to this level, but now I'm clueless about the 'why' ... ?
Using the "Zoom" feature (ShiftF2) in Access revealed that the field actually did contain
Myriam
Myriam
This was probably due to a user accidentally hitting CtrlEnter during data entry, which added the newline and made the field look empty, so they typed the name in a second time.
I'm working on a flow where I get CSV files. I want to put the records into different directories based on the first field in the CSV record.
For ex, the CSV file would look like this
country,firstname,lastname,ssn,mob_num
US,xxxx,xxxxx,xxxxx,xxxx
UK,xxxx,xxxxx,xxxxx,xxxx
US,xxxx,xxxxx,xxxxx,xxxx
JP,xxxx,xxxxx,xxxxx,xxxx
JP,xxxx,xxxxx,xxxxx,xxxx
I want to get the field value of the first field i.e, country. Put those records into a particular directory. US records goes to US directory, UK records goes to UK directory, and so on.
The flow that I have right now is:
GetFile ----> SplitText(line split count = 1 & header line count = 1) ----> ExtractText (line = (.+)) ----> PutFile(Directory = \tmp\data\${line:getDelimitedField(1)}). I need the header file to be replicated across all the split files for a different purpose. So I need them.
The thing is, the incoming CSV file gets split into multiple flow files with the header successfully. However, the regex that I have given in ExtractText processor evaluates it against the splitted flow files' CSV header instead of the record. So instead of getting US or UK in the "line" attribute, I always get "country". So all the files go to \tmp\data\country. Help me how to resolve this.
I believe getDelimitedField will only work off a singular line and is likely not moving past the newline in your split file.
I would advocate for a slightly different approach in which you could alter your ExtractText to find the country code through a regular expression and avoid the need to include the contents of the file as an attribute.
Using a regex of ^.*\n+(\w+) will capture the first line and the first set of word characters up to the comma and place them in the attribute name you specify in capture group 1. (e.g. country.1).
I have created a template that should get the value you are looking for available at https://github.com/apiri/nifi-review-collateral/blob/master/stackoverflow/42022249/Extract_Country_From_Splits.xml
Win XP, Excel 2007
I know there are various other posts on csv formatting but couldn't quite find what i needed.
Some of our data is held off site by another company and they send us a csv file every morning with the previous days data.
The problem is this data has come from web input forms that may have drop-down lists.
For example there may be a drop down list of Number of Employees with options like 1-10, 11-25, 26-50 etc
When we open the csv file in Excel certain options like 1-10 has been turned into Oct-01 date format which we do not want.
Is there an easy way to change these back OR reformat the cells and do a find...replace? (This didn't seem to work terribly well as it kept reverting back to the date)
Indeed is there a better way of opening the csv file to keep the formatting intact? and save us doing lots of find...replaces.
Ultimately we will need to open the csv in Excel though.
Grateful for any hints
Isn't that SO annoying? Here's how I deal with this issue:
When you open the CSV file in Excel, you should get a dialog with parsing options. First you select delimited or fixed then you get a screen that previews the data parsing.
It's easy to miss, but in the upper right corner of the dialog box there's an option to set a specific data format for each column. Select the column you want to protect and set the format to text. (This keeps Excel from dropping the leading zeros in ZIP codes for New England too!)
Once you get it into Excel, you can do a vlookup or replace to reset the values to your own codes.
Hope this helps. Good luck.