I try to import csv to elasticsearch using Logstash
Field data contains data in dd/mm/yy format - 08/02/23 for example
I need to change timestamp to date-month-year only
Try to use this:
date {
match => ["date", "dd/MM/yy"]
target => ["#timestamp"]
}
But logstash( or elastic) add hour:minute:seconds to my date
Is it possible to change timestamp field only to date-month-year,without time?
Logstash adds a #timestamp field automatically when the event enters event pipeline. But you're completely free to ignore that field altogether and use any other date field in your queries. Just make sure that your date is really mapped as a date type in your index mapping and you're good to go
Related
I use time as datatype in my rails migration but when I am retrieving it in model
it includes date. I wanted to be able to compute time difference without date.
Any way I can do to make my model return time only ?
I wanted this 03:15:00 not this 2000-01-01 05:54:42
If you want to convert a Datetime object into a Time object, you can use Time.parse.
date_and_time = Time.now # 2000-01-01 05:54:42
time_only = Time.parse(date_and_time) # 05:54:42
If you use the time data type for a column in your database migration you should be fine. Rails fills the missing fields (in this case year, month, day) with default values since the data type requires them. Since all responses get these default values you should be able to calculate time differences without date since all values have the same default value. If you want to get the time as string back in the format you want you can use #strftime method.
Eg:
time.strftime('%H:%M:%S')
I want to transform the value of fields name and Date using pentaho(kattle) and store it in standard format in any table.
For example
id,name,f_n,Date
1,j_vick,03-05-2015
2,jo_vick,04,08,2016
3,Arn_jonnn_vick,05,07,2017
Now I want to transform it using pentaho(kattle) IDE and store it in data base like below:
id,name,Date
1,john_vick,03/05/2015
2,john_vick,04/08/2016
3,Arn_john_vick,05/07/2017
I don't want the transformation steps concern with extraction database storage.
The date format is just some clothing of a value. Just read it with Kettle and store it as is. The database will store it in its proper internal binary format anyway.
Unless, you have to store them explicitly as varchar. In that case, use the Metadata tab of the Select Value step. Define your column as Date and specify the format as dd/MM/yyyy or MM/dd/yyyy. It will be kept internally as a Date, and converted into a String with the selected format at the last moment.
You also seam to have an other problem: the day, month and year are in three columns. The easiest way is to use a Modified Java Script Value step, in which you define a new column date = new Date(year, month, year), with type Date, and let Kettle handle with the format.
Maybe you have mixed input, in which case you can use a Filter or a Swtich step, based on weather you have something in the day and month column.
When that is done, you can make a job that runs the transformation on all the tables. You have an example in the sample/jobs/process all tables furnished with the Pentaho Data Integrator (aka Kettle).
I'm trying to import CSV data into MySQL database. I have four Date fields. when the import finishes, all my dates are zeroes: .
I've set the date format to DATE in Excel prior to saving it as CSV. I've set the table field as DATE in MySQL.
I'm using phpMyAdmin to do the import.
Mysql only excepts dates in YYYY-MM-DD format.
The Date format in excel is 06/09/2015. Thus, not in the format that mysql is expecting. You must change this to a format that is accepted in mysql otherwise all your date fields will appear as 0000-00-00.
In order to change the date format in excel: right click on the top cell. Choose format cells from the drop down list. change the local to something like 'Afrikans'. Choose the format that looks like 2001-03-14. Use the top cell to fill down. Then save the document.
Just a quick note: Excel sometimes tries to do too much and will revert this column back to a the English(U.S) default time zone. So, if you plan on doing more editing make sure that the column has not reverted back.
Here is a link to more string literals on dev.mysql.
I have used PHPExcel library to import an Excel file (.xlsx) and insert into MySQL. I'm facing two problems with the date field:
1- In the excel sheet I try to change the date format to the desired one. Upon saving, the cells show the format I need (yyyy/mm/dd) but when I double-click to edit them they again change back to how it was before (mm/dd/yyyy).
2- When the file is imported to MySQL through PHPExcel all the columns are inserted correctly except the date column which inserts 0000-00-00. The format in the Excelsheet is the same as in MySQL but it enters is all zeros.
Please help!
The value store in PHPExcel is an Excel serialized datetime value (a float, representing the number of days since 1/1/1900 or 1/1/9004 depending on whether Windows or Mac Calendar was used). Changing the format is simply changing the way this value is displayed.
You will either need to know in advance whether a cell contains a date, or test it using the PHPExcel_Shared_Date::isDateTime() method, and convert it appropriately. You can then either retrieve it using getFormattedValue() to return the value as a formatted string; or use the built-in conversion functions to convert it to a unix timestamp (PHPExcel_Shared_Date::ExcelToPHP()) or a PHP DateTime object (PHPExcel_Shared_Date::ExcelToPHPObject()) that you can then format using the standard PHP date formatting functions before using it in your MySQL INSERT/UPDATE statements.
Change the date format to yyyy-mm-dd, save the file as CSV and import that into your database.
I have one problem in my tool. I am using one MySQL table called calendar. From this table I am fetching some date values. The format of date in calendar table is yyyy-mm-dd. But in my system having dd-mmm-yy. In my c# application I fetched the date from the Calendar and If I displayed the date in c# application means date format(2012-10-05) changed to system format(05-oct-12). But I need to display in yyyy-mm-dd format. I don't know how to solve this problem.
To display DateTime in specific format, you can specify the format in .ToString() method like:
DateTime dt = DateTime.Now; //your DateTime variable
Console.WriteLine(dt.ToString("yyyy-mm-ddd"));
You may see the article: Standard Date and Time Format Strings - MSDN