I am using the Extension:JsonConfig on my docker instance of wikidata that has some tables loaded onto it. The configuration for the extension in my LocalSettings.php is as follows,
$wgJsonConfigEnableLuaSupport = true;
$wgJsonConfigModels['Tabular.JsonConfig'] = 'JsonConfig\JCTabularContent';
$wgJsonConfigs['Tabular.JsonConfig'] = [
'namespace' => 486,
'nsName' => 'Data',
// page name must end in ".tab", and contain at least one symbol
'pattern' => '/.\.tab$/',
'license' => 'CC0-1.0',
'isLocal' => true,
'store' => true,
];
When i query the local instance using the following url,
http://<DOMAIN_HERE>/w/api.php?action=query&list=search&srsearch=tab contentmodel:Tabular.JsonConfig &srnamespace=486&srlimit=10&format=json
i receive the following response
{"batchcomplete":"","limits":{"search":10},"query":{"searchinfo":{"totalhits":0},"search":[]}}
which means that no matches have been found even though tables that match the query statement do exist.
This same query works with commons database when the following is done
https://commons.wikimedia.org/w/api.php?action=query&list=search&srsearch=tab%20contentmodel:Tabular.JsonConfig%20&srnamespace=486&srlimit=10&format=json
Can anyone point me out as to what i am doing wrong here?
Related
I'm scraping a shoutbox which is limited to 10 messages; it's asynchronous and when the 11th item appears the first one is gone.
I set up a puppeteer, it scrapes the structure correctly as an array, which I dump to mongodb. The easiest way automating this I came up with is running script with the watch command and static interval.
The question is how to skip duplicates items in log, items shouldn't be unique, just don't dump the same twice. And there's probably a better way to cycle this process.attached screenshot
You can use db.collection.distinct() in MongoDB to obtain the distinct messages from your database:
db.messages.distinct( 'message' );
Alternatively, you can use db.collection.createIndex() to create a unique index in your database so that the collection will not accept insertion or update of a document where the index key value matches an existing value in the index:
db.messages.createIndex( { 'message' : 1 }, { 'unique' : true } );
In your Puppeteer script, you can use page.evaluate() in conjunction with the Set object to obtain distinct messages from the web page that you are scraping:
const distinct_messages = await page.evaluate( () => new Set( Array.from( document.querySelectorAll( '.message' ), e => e.textContent ) ) );
Any one know how to send error messages to database in laravel which generate from app/exceptions/handler.php ?
I need to send what error massages generated in report() method to database.
If you are interested doing this manually, you can do something as following.
Step 1 -
Create a model to store errors that has a DB structure as following.
class Error extends Model
{
protected $fillable = ['user_id' , 'code' , 'file' , 'line' , 'message' , 'trace' ];
}
Step 2
Locate the App/Exceptions/Handler.php file, include Auth, and the Error model you created. and replace the report function with the following code.
public function report(Exception $exception) {
// Checks if a user has logged in to the system, so the error will be recorded with the user id
$userId = 0;
if (Auth::user()) {
$userId = Auth::user()->id;
}
$data = array(
'user_id' => $userId,
'code' => $exception->getCode(),
'file' => $exception->getFile(),
'line' => $exception->getLine(),
'message' => $exception->getMessage(),
'trace' => $exception->getTraceAsString(),
);
Error::create($data);
parent::report($exception);
}
(I am demonstrating this using laravel 5.6)
Because Laravel uses Monolog for handling logging it seems that writing Monolog Handler would be the cleanest way.
I was able to find something that exists already, please have a look at monolog-mysql package. I did not use it, so I don't know whether it works and if it works well, but it's definitely good starting point.
What I am trying to do is to cache all the results in a MySQL table that seldom changes, so as to minimize calls to database and increase query speed. There are about 100k records in there.
Is there a library that can sync changes made in this table, like say when a record is updated or inserted, the redis cache will also be invalidated and updated.
I have seen one for elasticsearch, but nothing for redis.
From this page:
Yii copying data from one model to another
There is this comment:
You can get all models attributes by:
$data = $model->attributes;
and assign them to another model
$anotherModel = new AnotherActiveRecord();
$anotherModel->setAttributes($data);
now another model will extract whatever it can from $data
I'm curious, can a Redis cache also "mirror" the data from a database table in a similar way?
Or is this just a bad idea overall, and its better off caching the query as it comes along, or is there a better way.
You can enable caching based on https://www.yiiframework.com/doc/guide/2.0/en/caching-data
[
'components' => [
'cache' => [
'class' => 'yii\redis\Cache',
'redis' => [
'hostname' => 'localhost',
'port' => 6379,
'database' => 0,
]
],
],
]
and then use Query Caching which natively defined on query builder level
$result = $db->cache(function ($db) {
// the result of the SQL query will be served from the cache
// if query caching is enabled and the query result is found in the cache
// ... perform SQL queries here ...
});
Also you can use Cache Dependencies based on your table (some criteria like if max(updated_at) is changed or not).
// Create a dependency on updated_at field
$dependency = new yii\caching\DbDependency(['sql' => 'select max(updated_at) from my_table']);
$duration = 60; // cache query results for 60 seconds.
$result = $db->cache(function ($db) {
// ... perform SQL queries here ...
return $result;
}, $duration, $dependency);
I get an Error-Message in a UnitTest in CakePHP 3.2 and the official documentation doesn't help me here anymore. I think the error has something todo with the SQL-Joins I try to use.
The Error-Message is the following:
`1) App\Test\TestCase\Controller\GetContentControllerTest::testIndex
PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'contentmapper_test.CmDeviceclasses' doesn't exist`
In my Testclass GetContentControllerTest I load my fixtures that I need and that creates my Database-Tables on start:
`public $fixtures = [
'app.cm_content_options',
'app.cm_content_addresses',
'app.cm_deviceclasses',
'app.cm_properties'
];`
In the setUp()-Method I load the Main-Table:
`public function setUp()
{
parent::setUp();
$this->CmContentOptions = TableRegistry::get('CmContentOptions');
}`
My Test-Method testIndex() looks like this:
public function testIndex()
{
//find the belonging ContentOption to address data
//submitted by the client
$this->testFindAllByUriAndDeviceclassAndBoxId();
assert($this->arrObjContentOptions->count() == 1);
}
The testFindAllByUriAandDeviceclassAndBoxID() looks like shown in the following Image (the Editor is not able to prettyprint it correctly):
testFindAllByUriAandDeviceclassAndBoxID()
It's hard to describe the whole Context; I hope it is possible to understand.
The Error happens exactly on the statement shown in the image:
$result = $query->toArray()
I think I just forgot something to add in the setUp() Method or something like that.
I hope anyone can help.
You joins are set up incorrectly, you're mixing up aliases and table names.
The alias is the key of the join array, and the table key should hold the actual database table name, not the table class name.
Given that you are following CakePHPs naming conventions for your database table names, your join setup should look more like this
[
'CmDeviceclasses' => [ /* < this is the SQL alias */
'table' => 'cm_deviceclasses', /* < this is the database table name */
'type' => 'LEFT',
'conditions' => [
'CmDeviceclasses.classname' => $this->deviceclass
]
],
'CmContentAddresses' => [
'table' => 'cm_content_addresses',
'type' => 'INNER',
'conditions' => [
'CmContentAddresses.uri' => $this->uri,
'CmContentAddresses.boxid' => $this->boxid,
]
],
],
[
'CmDeviceclasses.classname' => 'string',
'CmContentAddresses.uri' => 'string',
'CmContentAddresses.boxid' => 'string'
]
There is no technical need to follow the CamelCase conventions for the aliases, but for sure it doesn't hurt to generally stick to the conventions.
ps, if you setup the associations properly, then there should be no need to use manual joins, you could just use Query::contain() and Query::innerJoinWith() or Query::matching().
See
Cookbook > Database Access & ORM > Associations - Linking Tables Together
Cookbook > Database Access & ORM > Retrieving Data & Results Sets > Retrieving Associated Data
Cookbook > Database Access & ORM > Retrieving Data & Results Sets > Filtering by Associated Data
i'am new to Databases and to DBIx:Class. So please forgive me if this is a total newbie fault.
I just followed a tutorial and then i tried to deploy the schema to my database. According to the tutorial i split the modules up in several files. After i ran createTable.pl 'mysqlshow bla' shows me a empty database.
Database is up and running. Creating a table via the mysql CREATE TABLE statement does work.
Skript file which should create a table according to the schema ../createTable.pl
#!/usr/bin/env perl
use Modern::Perl;
use MyDatabase::Main;
my ($database, $user) = ('bla', 'flo');
my $schema = MyDatabase::Main->connect("dbi:mysql:dbname=$database", "$user");
$schema->deploy( { auto_drop_tables => 1 } );
Main.pm for loading the namespaces ../MyDatabase/Main.pm
package MyDatabase::Main;
use base qw/ DBIx::Class::Schema /;
__PACKAGE__->load_namespaces();
1;
Schema file for the table ../MyDatabase/Result/Album.pm
package MyDatabase::Main::Result::Album;
use base qw/ DBIx::Class::Core /;
__PACKAGE__->load_components(qw/ Ordered /);
__PACKAGE__->position_column('rank');
__PACKAGE__->table('album');
__PACKAGE__->add_columns(albumid =>
{ accessor => 'album',
data_type => 'integer',
size => 16,
is_nullable => 0,
is_auto_increment => 1,
},
artist =>
{ data_type => 'integer',
size => 16,
is_nullable => 0,
},
title =>
{ data_type => 'varchar',
size => 256,
is_nullable => 0,
},
rank =>
{ data_type => 'integer',
size => 16,
is_nullable => 0,
default_value => 0,
}
);
__PACKAGE__->set_primary_key('albumid');
1;
I already spent some hours on finding help through google but there isn't much related to the deploy() method.
Can anyone explain me what my mistake is?
Thank you
You can find the documentation for all CPAN Perl modules on metacpan.org (newer, full-text indexed) and search.cpan.org.
Read the docs for DBI, you'll find an environment variable called DBI_TRACE that when set will print every SQL statement to STDOUT.
DBIx::Class has a similar called DBIC_TRACE.
The first one should help you to see what the deploy method is doing.
Is no password required for connecting to your database?
Ok today i played again with perl and database stuff and i found out what the mistake was.
First of all i started with DBI_TRACE and DBIC_TRACE, it produced a lot of messages but nothing i could handle, for me it seemed like nothing gave me a hint on the problem.
Then i searched google for a while about this problem and for more examples of the deploy method. At some point i noticed that my folder structure is wrong.
The Schema file for the table should be placed in
../MyDatabase/Main/Result/Album.pm
instead of being placed in
../MyDatabase/Result/Album.pm
After moving the Schema file to the correct folder everything worked well.
Shame on me for this mistake :( But thank you for your help