I have a list of e-mails in a text file and a Person table with an email column.
I need to know which of the emails are in the table, and which are not.
So I was thinking in creating a query and do some kind of left join in my raw e-mail data with the Person table.
Now, I can do this:
select count(*) from PERSON p where p.EMAIL in ("email1#mail","XXXX#mail.com");
But, what I want is to return something like this:
Raw_Email email
email1#mail.com email1#mail.com
XXXX#mail.com null
XXXXX XXXXX
Loading the data from the text file into some table is the best option (the table may be temporary). Loaded data may then be used by the common way.
Another option is in using CSV Engine - but this engine must be available on your server instance (check SHOW ENGINES output). You must create table using this engine joined to your file then use this table by the common way.
Anycase you must have CREATE TABLE privilege.
The last option is in use LOAD_FILE() function. File content will be loaded as one string literal which may be parsed and splitted on separate values (this is easy enough on recent MySQL version). But you must to have secure_file_privilege option set, and if it is not empty then the file must be placed into specified folder. This option does not need in special additional privileges. Alternatively this function can be used in stored procedure where loaded data will be used in dynamic SQL.
I don't think that MySQL supports direct access to text files -- at least without extensions. So, you should import the data into the database. The simplest method is LOAD DATA INFILE.
Then you can use a LEFT JOIN to do what you want:
select e.raw_email, p.email
from staging_emails e left join
persons p
on p.email = e.raw_email;
Related
My scenario is as followed.
From one process I retrieve data from a table.
id,user_name
1,sachith
2,nalaka
I need to retrieve account details from account_details table for these ids.
I have tried to use various database related processors. But none of them support flowfile content.
How can I retrieve records only for these id?
use below:
ExecuteSQL( account_details)
-> convertAvroToJSON
-> EvaluateJsonPath
->AttributesToJson
( here you take only id and ignore test)
Take a look at the LookupRecord using a DatabaseRecordLookupService controller service. That should allow you to use the id field to look up additional fields from a database and add them to the outgoing records. This is a common "enrichment" pattern, where the lookups can be done against databases, CSV files, etc.
You can use QueryRecord processor to query data from flowfiles. You will need to set a reader and a writer inside this processor to open your file properly and write as well. To create a query, you must create a property with the name of the query and put the query itself as the value for this property. After that, you can create an output stream for this property.
The query syntax is Apache Calcite.
You can find further explanation here
My situation is that I am creating a wordpress plugin, it creates a table in the directory on activation.
This table holds information entered into the plugin, when information is being entered the user has the option to upload images. The number they will choose to upload I do not know.
The issue I am having is figuring out how to add these URLS into the database, sure I can just put them in there but again I do not know how many URLS need to be added.
As I cannot use normalization with wordpress, how would I store the URLS in the DB. Say they upload 5 images but I do not have 5 separate columns (URL 1, URL 2...)
I should also note that these images will be fetched using a for loop, so each image will be sent off regardless of the number they are uploading.
Any help would be appreciated
Cheers.
Also, You can make additional table with:
id, information_id, filename, ordr
Firstly, insert into infromation_table, get last inserted ID
Then, insert into filenames_table all your filenames in order
Finally, you can select data:
SELECT i.id, i.title, GROUP_CONCAT(a.filename ORDER BY a.ordr ASC SEPARATOR ',') AS filenames
FROM information_table AS i
JOIN filenames_table AS a ON (a.information_id = i.id)
If you want to store all image urls or other data into one DB row, you must create an array with all the information that you need, and then using serialize you can put this array in one filed in the database.
This filed must be TEXT or LONGTEXT to be able to collect all this information.
Then, when you get data back, you must use unserialize and data will be converted to arrays again if needed.
This is how WP stores information in the postmeta table.
Note that in this way you cannot easily query the data, so if this is important to you maybe will be better to store each image on separate row.
Alright, I realize that what I am about to ask may not be possible, which I can live with. But If it is possible it will make my life a lot easier.
Within my MS Access Database, I am attempting to query 2 tables. These tables are both linked to my Access DB, one is linked to an Excel file (MSSB Reps DTP) containing a dump from a seperate DB2 database.
The other table in my query (SalesPage DNK Rep Query) is linked to another Access DB, which is in turn linked to a SQL databse. Here is an image of the Query design screen:
Where the tables are linked is not so much important as the fact that since they are linked tables, and since this is MS Access, I can not edit the tables. Therefore, I can not simply pop into design mode of either table and change the data type of a given column.
Each table has a column named CRD Number. I want to create an inner join between these two tables based on this column. Just a simple, everyday, inner join. I can not however, because the CRD Number column is stored as a Number for the MSSB Reps DTP table, and as text in the SalesPage DNK Rep Query table.
I was wondering if there is some way to temporarily use a fucntion to "cast" the CRD Number column fromo the MSSB Reps DTP table as text, so that I can run this query. However, any solution will be appreciated. Just don't tell me to edit the Excel document. I am trying very hard to avoid that for various reasons.
You can use CStr to cast the number as text.
SELECT *
FROM
[MSSB Reps DTP] AS m
INNER JOIN [SalesPage DNK Rep Query] AS s
ON CStr(m.[CRD Number]) = s.[CRD Number];
The Access query designer may refuse to display that join in Design View, but you can switch to SQL View and edit the statement text.
I want to try and keep this as one query and not use PHP, but it's proving to be tough.
I have a table called applications, that stores all the applications and some basic information about them.
Then, I have a table with all the types of applications in it, and that table contains a reference to another table which stores more specific data about the specific type of application in question.
select applications.id as appid, applications.category, type.title as type, type.id as tid, type.valuefld, type.tablename
from applications
left join type on applications.typeid=type.id
left join department on type.deptid=department.id
where not isnull(work_cat)
and work_cat != ''
and applications.deleted=0
and datei between '10-04-14' and '11-04-14'
order by type, work_cat
Now, in the old version, there is another query on every single result. Over hundreds of results... that sucks.
This is the query I'd like to integrate so I can get all the data in one result row. (Old is ASP, I'm re-writing it in PHP)
query = "select sum("&adors.fields("valuefld")&") as cost, description from "&adors.fields("tablename")&" where appid = '"&adors.fields("tablename")&"'"
Prepared statements, I'm aware, are the best solution, but for now they are not an option.
You can't do this with a plain SQL query - you need to have a defined set of tables that your query is based on. The fact that your current implementation queries from whatever table is named by tablename from the first result-set means that to get this all in one query, you will have to restructure your data. You have to know what tables you're querying from rather than having it dynamic.
If the reason for these different tables is the different information stored in each requiring different record (column) structures, you might want to look into Key/Value pair storage in a large table. Once you combine the dynamically named ones into a single location you can integrate your two queries together.
I'm very new to MySQL, although I've used SQL databases in other contexts before. I have a test site set up which has an online cPanel with access to phpMyAdmin. I'm attempting to setup a MySQL database, and so far it's working fine (I can connect to the Database and the table).
The only problem I'm having is with inserting data. I'd like to insert an entire array (specifically, the array will be a double[]) into one column. After looking at the column types available in phpMyAdmin, it doesn't seem to support inserting arrays other than Binary arrays.
I've found many solutions for inserting arrays programatically including this thread, but for this site we will be inserting data via the online cPanel. Is there a way to do that?
If you want access to that data, and want to be able to use the power of SQL to search in your double[], you should do it this way:
First, you should spend some time researching relational databases. They allow you to create linked data.
An important part of every relational database is using good keys. A key is a unique identifier for a row that allows you to access the data on that row in an efficient manner.
Another important part of relational databases are indexes. Indexes are not required to be unique. But are useful if you are trying to search on them (SQL has made an "index" of the table based on a column or group of columns)
If you wanted to create a table that would have a double[] array, you might instead create a 2nd table that relates to the first table by the first tables primary key.
CREATE TABLE base (
base_id INT AUTO_INCREMENT,
name VARCHAR(32),
PRIMARY KEY(base_id)
);
CREATE TABLE darray (
base_id INT,
data DOUBLE,
INDEX(base_id)
);
To get the information back out that you want, you can select using a JOIN statement. If you wanted to get all the information where the base_id was 3, you would write it like so:
SELECT * FROM base
JOIN darray ON darray.base_id = base.base_id
WHERE base.base_id = 3;
The advanced form of writing this with aliasing
SELECT * FROM base b
JOIN darray d ON d.base_id = b.base_id
WHERE b.base_id = 3;
If you don't want to have access to the data, but are just recalling it, you should do it this way: (Although this is debatable, I still recommend the above way, if you are willing to learn more sql)
I assume you will be using PHP, we will be serializing the data (see: http://php.net/manual/en/function.serialize.php)
Note we will don't have the darray table, but instead add a
data BLOB
to the base table.
Inserting with PHP serialized data
<?php
$serializedData = serialize($darray);
$result = mysql_query("INSERT INTO base (name, data) VALUES('a name', '$serializedData ')");
Getting the serialized data
<?php
$result = mysql_query("SELECT data FROM base WHERE base_id=3");
if($result && mysql_affected_rows($result) > 0) {
$serializedData = mysql_result($result, 0, 'data');
$darray = unserialize($serializedData);
}
You can import data for tables with a .sql file (basically just a file full of insertion queries) but phpMyAdmin doesn't support inserting data from arbitrary data types. If you want to insert a double[] array as multiple rows in a table, you'll need to take an approach similar to the one in the thread you linked.
(Note that you can always write such a program for the explicit purpose of generating a .sql file which you then use for deployment.)