Hook code :
user_receive_packet({Packet, #{jid := JID} = C2SState}) ->
?INFO_MSG("~p.", [Packet]),
{Packet, C2SState}.
Output:
{message,<<"49a85099-6fa6-4aa5-84ab-bced89adf847">>,groupchat,<<"en">>,{jid,<<"sgj279_59f990094dd64c4846cc38c5_59fa99bbfdbd1ab22e59df29">>,<<"conference.localhost">>,<<"localhost">>,<<"sgj279_59f990094dd64c4846cc38c5_59fa99bbfdbd1ab22e59df29">>,<<"conference.localhost">>,<<"localhost">>},{jid,<<"test1">>,<<"localhost">>,<<"49921480726720246332274">>,<<"test1">>,<<"localhost">>,<<"49921480726720246332274">>},[],[{text,<<>>,<<"{\"type\":\"text\",\"payload\":{\"body\":\"a beautiful\"}}">>}],undefined,[{mam_archived,{jid,<<"sgj279_59f990094dd64c4846cc38c5_59fa99bbfdbd1ab22e59df29">>,<<"conference.localhost">>,<<>>,<<"sgj279_59f990094dd64c4846cc38c5_59fa99bbfdbd1ab22e59df29">>,<<"conference.localhost">>,<<>>},<<"1549043294480251">>},{stanza_id,{jid,<<"sgj279_59f990094dd64c4846cc38c5_59fa99bbfdbd1ab22e59df29">>,<<"conference.localhost">>,<<>>,<<"sgj279_59f990094dd64c4846cc38c5_59fa99bbfdbd1ab22e59df29">>,<<"conference.localhost">>,<<>>},<<"1549043294480251">>}],#{ip => {0,0,0,0,0,0,0,1},mam_archived => true,muc_sender_real_jid => {jid,<<"test1">>,<<"localhost">>,<<"49921480726720246332274">>,<<"test1">>,<<"localhost">>,<<"49921480726720246332274">>},stanza_id => 1549043294480251}}.
I want to read both the stanza_id field value and the timestamp listed under mam_archived
what is the syntax to parse any field ?
Try:
?INFO_MSG("Stanza ID: ~p", [ maps:get(stanza_id, Packet#message.meta) ]),
Related
I need a brief explanation of how can I convert MySQL data to Elastic Search using logstash.
can anyone explain the step by step process about this
This is a broad question, I don't know how much you familiar with MySQL and ES. Let's say you have a table user. you may just simply dump it as csv and load it at your ES will be good. but if you have a dynamic data, like the MySQL just like a pipeline, you need to write a Script to do those stuff. anyway you can check the below link to build your basic knowledge before you ask How.
How to dump mysql?
How to load data to ES
Also, since you probably want to know how to covert your CSV to json file, which is the best suite for ES to understand.
How to covert CSV to JSON
You can do it using the jdbc input plugin for logstash.
Here is a config example.
Let me provide you with a high level instruction set.
Install Logstash, and Elasticsearch.
In Logstash bin folder copy jar ojdbc7.jar.
For logstash, create a config file ex: config.yml
#
input {
# Get the data from database, configure fields to get data incrementally
jdbc {
jdbc_driver_library => "./ojdbc7.jar"
jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
jdbc_connection_string => "jdbc:oracle:thin:#db:1521:instance"
jdbc_user => "user"
jdbc_password => "pwd"
id => "some_id"
jdbc_validate_connection => true
jdbc_validation_timeout => 1800
connection_retry_attempts => 10
connection_retry_attempts_wait_time => 10
#fetch the db logs using logid
statement => "select * from customer.table where logid > :sql_last_value order by logid asc"
#limit how many results are pre-fetched at a time from the cursor into the client’s cache before retrieving more results from the result-set
jdbc_fetch_size => 500
jdbc_default_timezone => "America/New_York"
use_column_value => true
tracking_column => "logid"
tracking_column_type => "numeric"
record_last_run => true
schedule => "*/2 * * * *"
type => "log.customer.table"
add_field => {"source" => "customer.table"}
add_field => {"tags" => "customer.table" }
add_field => {"logLevel" => "ERROR" }
last_run_metadata_path => "last_run_metadata_path_table.txt"
}
}
# Massage the data to store in index
filter {
if [type] == 'log.customer.table' {
#assign values from db column to custom fields of index
ruby{
code => "event.set( 'errorid', event.get('ssoerrorid') );
event.set( 'msg', event.get('errormessage') );
event.set( 'logTimeStamp', event.get('date_created'));
event.set( '#timestamp', event.get('date_created'));
"
}
#remove the db columns that were mapped to custom fields of index
mutate {
remove_field => ["ssoerrorid","errormessage","date_created" ]
}
}#end of [type] == 'log.customer.table'
} #end of filter
# Insert into index
output {
if [type] == 'log.customer.table' {
amazon_es {
hosts => ["vpc-xxx-es-yyyyyyyyyyyy.us-east-1.es.amazonaws.com"]
region => "us-east-1"
aws_access_key_id => '<access key>'
aws_secret_access_key => '<secret password>'
index => "production-logs-table-%{+YYYY.MM.dd}"
}
}
}
Go to bin, Run as
logstash -f config.yml
I have the following configuration for the formatter component in my project:
'formatter' => [
'dateFormat' => 'dd.MM.yyyy',
'datetimeFormat' => 'dd.MM.yyyy HH:mm',
'decimalSeparator' => '.',
'thousandSeparator' => ' ',
'currencyCode' => 'EUR',
],
Now, for my datetimeFormat I'd like to suffix it with an h letter/string at the end of it.
I have tried with:
'datetimeFormat' => 'dd.MM.yyyy HH:mm\\h',
But the escaped \\h returns a letter g there. I know that \\h works with date.
How do we do escapes with Yii 2?
Okay, found it while reading formatter documentation and ICU manual.
To achieve this, escape the letter with wrapping your letter/string with single quotes:
'datetimeFormat' => "dd.MM.yyyy HH:mm'h'",
Sample output: 30.12.1990 12:25h.
I'm trying to use eloquent to get me a grouped by response and at the same time give me a Pagination response (The one that gives me the link to the second page).
I'm trying to do this:
App\Eating::Where('student_id', 2)->orderBy('created_at', 'DESC')->groupBy(function ($row) {
return Carbon\Carbon::parse($row->created_at)->format('Y-m-d');
})->paginate(25);
But, I'm getting this error when running it in the Tinker:
PHP warning: strtolower() expects parameter 1 to be string, object given in D:\Folder\vendor\laravel\framework\src\Illuminate\Database\Grammar.php on line 58
without the groupBy, I'm getting the correct result:
>>> App\Eating::Where('student_id', 2)->orderBy('created_at', 'DESC')->paginate(25)->toArray();
=> [
"total" => 1,
"per_page" => 25,
"current_page" => 1,
"last_page" => 1,
"next_page_url" => null,
"prev_page_url" => null,
"from" => 1,
"to" => 3,
"data" => [
[
"id" => 5,
"status" => "Comeu Bem",
"created_at" => "2017-07-05 13:55:25",
"updated_at" => "2017-07-05 13:55:25",
],
],
]
BUT, when I remove the pagination, I do get the error but only because I added the get():
>>> App\Eating::Where('student_id', 2)->orderBy('created_at', 'DESC')->groupBy(function ($row) {
... return Carbon\Carbon::parse($row->created_at)->format('Y-m-d');
... })->get();
PHP warning: strtolower() expects parameter 1 to be string, object given in D:\Joao\git\F1Softwares\Code\Server\F1Softwares\vendor\laravel\framework\src\Illuminate\Database\Grammar.php on line 58
>>>
>>>
>>> App\Eating::Where('student_id', 2)->orderBy('created_at', 'DESC')->groupBy(function ($row) {
... return Carbon\Carbon::parse($row->created_at)->format('Y-m-d');
... });
=> Illuminate\Database\Eloquent\Builder {#855}
Any idea what I could be doing wrong? I do need to have the orderBy AND the pagination, to make it easier for the app to show the results(It is a RestFul call).
Thanks,
João
You must call the groupBy() method on a collection, but it seems this won't work with paginate(). You could try using the forPage() method on the collection:
App\Eating::where('student_id', 2)->orderBy('created_at', 'DESC')
->get()->groupBy(function ($eating) {
return $eating->created_at->format('Y-m-d');
})->forPage(1, 25);
Also, just a note, you don't need to use Carbon to parse the date, Eloquent does this for you.
Alternatively, you could try to manually create your paginator once you have the collection grouped using Illuminate\Pagination\LengthAwarePaginator.
$eatings = App\Eating::where('student_id', 2)->orderBy('created_at', 'DESC')
->get()->groupBy(function ($eating) {
return $eating->created_at->format('Y-m-d');
});
$paginatedEatings = new LengthAwarePaginator($eatings, $eatings->count(), 25);
return $paginatedEatings->toArray();
I installed the above extension via composer and follow the documentation for every step; in my :
view : use kartik\datecontrol\DateControl;// <?=$form->field($model, 'dated')->widget(DateControl::classname(), [
'type'=>DateControl::FORMAT_DATE,
'ajaxConversion'=>false,
'options' => ['pluginOptions' => ['autoclose' => true ],'class'=>'col-xs-12 form-control input-sm']])?>
Web.php - Module configuration :
use \kartik\datecontrol\Module;
'datecontrol' => [
'class' => 'kartik\datecontrol\Module',
// format settings for displaying each date attribute (ICU format example)
'displaySettings' => [
Module::FORMAT_DATE => 'php:dd-m-Y',
Module::FORMAT_TIME => 'php:H:i:s',
Module::FORMAT_DATETIME => 'php:Y-m-d H:i:s',
],
// format settings for saving each date attribute (PHP format example)
'saveSettings' => [
Module::FORMAT_DATE => 'php:Y-m-d',
Module::FORMAT_TIME => 'php:H:i:s',
Module::FORMAT_DATETIME => 'php:Y-m-d H:i:s',
],
// set your display timezone
// 'displayTimezone' => 'Asia/Kolkata',
// set your timezone for date saved to db
// 'saveTimezone' => 'UTC',
// automatically use kartik\widgets for each of the above formats
'autoWidget' => true,
// default settings for each widget from kartik\widgets used when autoWidget is true
'autoWidgetSettings' => [
Module::FORMAT_DATE => ['type'=>2, 'pluginOptions'=>['autoclose'=>true]], // example
Module::FORMAT_DATETIME => [], // setup if needed
Module::FORMAT_TIME => [], // setup if needed
],
// custom widget settings that will be used to render the date input instead of kartik\widgets,
// this will be used when autoWidget is set to false at module or widget level.
'widgetSettings' => [
Module::FORMAT_DATE => [
'class' => 'yii\jui\DatePicker', // example
'options' => [
'dateFormat' => 'php:d-M-Y',
'options' => ['class'=>'form-control'],
]
]
]
// other settings
]
When I run the view I got the following error message :
{"name":"Invalid Configuration","message":"The class
'\kartik\date\DatePicker' was not found and is required for
DateControl 'date' format.\n\nPlease ensure you have installed one of
'yii2-widgets' OR 'yii2-widget-datepicker' extensions. To install, you
can run this console command from your application root:\n\nphp
composer.phar require kartik-v/yii2-widgets: \"#dev\"\n\n--- OR
---\n\nphp composer.phar require kartik-v/yii2-widget-datepicker: \"#dev\"","code":0,"type":"yii\base\InvalidConfigException","file":"C:\wamp\www\pub\vendor\kartik-v\yii2-krajee-base\Config.php","line":118,"stack-trace":["#0
C:\wamp\www\pub\vendor\kartik-v\yii2-krajee-base\Config.php(195):
kartik\base\Config::checkDependency('\\kartik\\date\\Da...',
Array, 'for DateControl...')","#1
C:\wamp\www\pub\vendor\kartik-v\yii2-datecontrol\DateControl.php(215):
kartik\base\Config::validateInputWidget('\\kartik\\date\\Da...',
'for DateControl...')","#2
C:\wamp\www\pub\vendor\kartik-v\yii2-datecontrol\DateControl.php(154):
kartik\datecontrol\DateControl->initConfig()","#3
C:\wamp\www\pub\vendor\yiisoft\yii2\base\Object.php(107):
kartik\datecontrol\DateControl->init()","#4 [internal function]:
yii\base\Object->__construct(Array)","#5
C:\wamp\www\pub\vendor\yiisoft\yii2\di\Container.php(372):
ReflectionClass->newInstanceArgs(Array)","#6
C:\wamp\www\pub\vendor\yiisoft\yii2\di\Container.php(151):
yii\di\Container->build('kartik\\datecont...', Array, Array)","#7
C:\wamp\www\pub\vendor\yiisoft\yii2\BaseYii.php(344):
yii\di\Container->get('kartik\\datecont...', Array, Array)","#8
C:\wamp\www\pub\vendor\yiisoft\yii2\base\Widget.php(97):
yii\BaseYii::createObject(Array)","#9
C:\wamp\www\pub\vendor\yiisoft\yii2\widgets\ActiveField.php(665):
yii\base\Widget::widget(Array)","#10
C:\wamp\www\pub\views\activite\schedules.php(49):
yii\widgets\ActiveField->widget('kartik\\datecont...',
Array)","#11
C:\wamp\www\pub\vendor\yiisoft\yii2\base\View.php(325):
require('C:\\wamp\\www\\pub...')","#12
C:\wamp\www\pub\vendor\yiisoft\yii2\base\View.php(247):
yii\base\View->renderPhpFile('C:\\wamp\\www\\pub...',
Array)","#13
C:\wamp\www\pub\vendor\yiisoft\yii2\base\View.php(149):
yii\base\View->renderFile('C:\\wamp\\www\\pub...', Array,
Object(app\controllers\ActiviteController))","#14
C:\wamp\www\pub\vendor\yiisoft\yii2\base\Controller.php(371):
yii\base\View->render('schedules', Array,
Object(app\controllers\ActiviteController))","#15
C:\wamp\www\pub\controllers\ActiviteController.php(407):
yii\base\Controller->render('schedules', Array)","#16 [internal
function]:
app\controllers\ActiviteController->actionAddsch('4')","#17
C:\wamp\www\pub\vendor\yiisoft\yii2\base\InlineAction.php(55):
call_user_func_array(Array, Array)","#18
C:\wamp\www\pub\vendor\yiisoft\yii2\base\Controller.php(151):
yii\base\InlineAction->runWithParams(Array)","#19
C:\wamp\www\pub\vendor\yiisoft\yii2\base\Module.php(455):
yii\base\Controller->runAction('addsch', Array)","#20
C:\wamp\www\pub\vendor\yiisoft\yii2\web\Application.php(84):
yii\base\Module->runAction('activite/addsch', Array)","#21
C:\wamp\www\pub\vendor\yiisoft\yii2\base\Application.php(375):
yii\web\Application->handleRequest(Object(yii\web\Request))","#22
C:\wamp\www\pub\web\index.php(12):
yii\base\Application->run()","#23 {main}"]}
Try installing the following widgets:
yii2-widget-datepicker
yii2-widget-datetimepicker
or better, install the kartik widgets using composer:
php composer.phar require kartik-v/yii2-widgets "*"
that will install the following widgets:
yii2-krajee-base
yii2-widget-activeform
yii2-widget-affix
yii2-widget-alert
yii2-widget-colorinput
yii2-widget-datepicker
yii2-widget-datetimepicker
yii2-widget-depdrop
yii2-widget-fileinput
yii2-widget-growl
yii2-widget-rangeinput
yii2-widget-rating
yii2-widget-select2
yii2-widget-sidenav
yii2-widget-spinner
yii2-widget-switchinput
yii2-widget-timepicker
yii2-widget-touchspin
yii2-widget-typeahead
I need to get a readable JSON object from a PERL script but it's not in a readable format.
This is the code that produces the JSON.
while (my ($orderID, $possessorName, $itemDescription, $customerPickUpTime, $customerDropOffTime, $paymentAmount, $originAddress1, $originAddress2, $originNeighborhood, $originZipCode, $destinationAddress1, $destinationAddress2, $destinationNeighborhood, $destinationZipCode) = $sth->fetchrow_array)
{
%data = (orderID => $orderID, possessorName => $possessorName, itemDescription => $itemDescription, customerPickUpTime => $customerPickUpTime, customerDropOffTime => $customerDropOffTime, paymentAmount => $paymentAmount, originAddress1 => $originAddress1, originAddress2 => $originAddress2, originNeighborhood => $originNeighborhood, originZipCode => $originZipCode, destinationAddress1 => $destinationAddress1, destinationAddress2 => $destinationAddress2, destinationNeighborhood => $destinationNeighborhood, destinationZipCode => $destinationZipCode);
$json_obj = JSON->new->allow_nonref;
my $json_text = $json_obj->pretty->encode(\%data);
$query_results{"job$index"} = {"data" => $json_text};
$index++;
}
return $json_obj->pretty->encode(\%query_results, {ascii => 1, pretty => 1});
Everything works except when I look into the file (here's the printing line):
open (resultsFile, ">", "json_file.txt") || die "This doesn't work.";
print resultsFile "Results: \n\n $results";
The results are as follows:
{
"job3" : {
"data" : "{\n \"originAddress1\" : \"101 East 105th Street\",\n \"destinationZipCode\" : \"10128\",\n \"destinationNeighborhood\" : \"Upper East Side\",\n \"customerDropOffTime\" : \"2013-01-22 23:41:37\",\n \"originAddress2\" : \"\",\n \"paymentAmount\" : \"19.00\",\n \"customerPickUpTime\" : \"2013-01-22 22:56:37\",\n \"itemDescription\" : \"body\",\n \"destinationAddress1\" : \"180 East 93rd Street\",\n \"destinationAddress2\" : \"\",\n \"possessorName\" : \"Lisa Howard\",\n \"originZipCode\" : \"10029\",\n \"originNeighborhood\" : \"East Harlem\",\n \"orderID\" : \"723\"\n}\n"
},
The JSON object is formed correctly but the \n is the problem. It's not outputting with an actual newline. That's the issue.
This is because, $json_text is a string and not a hash. If you want to encode the whole thing as JSON, you must create an appropriate data structure
$query_results{"job$index"} = {"data" => \%data};
and give that as a whole to encode.