I am using KeyValueTextInputFormat for reading/processing a comma separated file :
100,56
89,586
123,68
However I get all the value in key, value field is coming as null, even after giving separator as comma (,). It is not picking the separator, not sure what is the issue,, here is my driver code:
Configuration conf = new Configuration();
conf.set("key.value.separator.in.input.line", ",");
Job job = new Job(conf, "citation data");
job.setJarByClass(Citation.class);
job.setJobName("citation data");
job.setMapperClass(MapClass.class);
job.setReducerClass(ReduceClass.class);
job.setInputFormatClass(KeyValueTextInputFormat.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);
KeyValueTextInputFormat.addInputPath(job, new Path("input/sample.txt"));
FileOutputFormat.setOutputPath(job, new Path("output2"));
System.exit(job.waitForCompletion(true)?0:1);
It works perfectly fine for me. In one of my mapred code, I just changed the following and it worked :
Changed the InputFormatClass to use KeyValueTextInputFormat
Added the config - conf.set("key.value.separator.in.input.line", ",");
Made sure than in the mapper has a look of something like Mapper<Text,Text,K,V>, which would make the map() method's signature to be something like this:
public void map(Text key, Text value, OutputCollector output, Reporter reporter)
throws IOException {}
No other change is required and you must get the first column's data as key and second column as value.
I guess the only thing you might have missed is the point 3.
Related
Trying to replicate the following MySQL query in Hibernate using CriteriaBuilder. This query adds first and last name, removes all whitespaces in between and search for results that matches the given string.
select * from users where replace(concat(first_name, last_name), " ", "") like 'jamesbon%';
final CriteriaBuilder criteriaBuilder = getCurrentSession().getCriteriaBuilder();
final CriteriaQuery<UserImpl> userCriteriaQuery = criteriaBuilder.createQuery(UserImpl.class);
final Root<UserImpl> userRoot = userCriteriaQuery.from(UserImpl.class);
// criteriaBuilder.concat(userRoot .get("firstName"), userRoot .get("lastName"))
Concat is available through the builder, so all you really need to add is the replace function.
What you need to do is create a class that implements org.hibernate.boot.spi.MetadataBuilderInitializer and use it to register the functions with Hibernate. Let's say your class is called com.project.hibernate.MetaContrib
package com.project.hibernate;
import org.hibernate.boot.MetadataBuilder;
import org.hibernate.boot.registry.StandardServiceRegistry;
import org.hibernate.boot.spi.MetadataBuilderInitializer;
import org.hibernate.dialect.function.StandardSQLFunction;
import org.hibernate.type.StringType;
public class MetaContrib implements MetadataBuilderInitializer {
#Override
public void contribute(MetadataBuilder metadataBuilder, StandardServiceRegistry serviceRegistry) {
metadataBuilder.applySqlFunction("str_replace", new StandardSQLFunction("replace", StringType.INSTANCE));
metadataBuilder.applySqlFunction("regex_replace", new StandardSQLFunction("REGEXP_REPLACE", StringType.INSTANCE));
}
}
The next step is to tell Hibernate to load this, by creating a file in the META-INF/services directory in your resources, called org.hibernate.boot.spi.MetadataBuilderInitializer. If such a directory doesn't exist, create it. The file has to contain the full name of the implementing class, and end in a new line.
Finally to use it:
expr1 = criteriaBuilder.concat(userRoot.get("firstName"), userRoot.get("lastName"));
expr2 = criteriaBuilder.function("str_replace", String.class, expr1, " ", "");
expr3 = criteriaBuilder.like(expr2, cb.parameter(String.class, "sv"));
userCriteriaQuery.where(expr3)
return createQuery(userCriteriaQuery)
.setParameter("sv", "jamesbon%")
.getResultList();
Detailed explanation:
The CriteriaBuilder creates a JPQL query. Your function expression becomes something like:
... WHERE function('str_replace', concat(u.firstName, u.lastName), ' ', '') LIKE :sv
Which when rendered to a native query will look like:
where replace(concat(u0.first_name, u0.last_name), ' ', '') like :sv
The function was registered under the name str_replace in JPQL, but it can be any name you choose. It's the name you give to the StandardSQLFunction constructor that tells it what the native name is.
Then further down the :sv internally becomes a ?, and when you use setParameter it tells the JDBC driver to safely send the string at that position.
However if you want to remove all whitespace, instead of merely all 0x20 space characters, you should use a regular expression like \s+ with the other function I put in MetaContrib. You can only do so if your MySQL is 8.0.4 or newer, or MariaDB 10.0.8 or newer. That function exists in the MariaDB10Dialect, so if you are using MariaDB, you may not need the MetaContrib class.
I am trying to do an easy search on a table that can be on any kind of database. The following query is working an the most databases, but I cannot find a solution which works on mysql.
The tables in my database are generated by the active objects framework, so I cannot change the names or config of those instances.
Here is the query that works fine on all databases but MySQL:
select * from "AO_69D057_FILTER" where "SHARED" = true AND "CONTAINS_PROJECT" = true AND UPPER("FILTER_NAME") like UPPER('%pr%').
MySql is not able to use the table name in double quotes for some reason. If I use the unquoted table name it works on MySQL but not on Postgres. Postgres is converting the table name to lowercase because it is unquoted. AO is generating the table names in upper case.
I also tried to use an alias, but that can not work because of the evaluation hierarchy of the statement.
Any suggestions how to get rid of the table name problem?
By default double quotes are used to columns.
You can change it:
SET SQL_MODE=ANSI_QUOTES;
Here is the documentation about it:
http://dev.mysql.com/doc/refman/5.7/en/sql-mode.html
I had the same problem. I select the query according to the exception I get. In the first call of the db search, I try without quotes if it fails then I try with quotes. Then I set useQueryWithQuotes variable accordingly so that in future calls I do not need to check the exception. Below is the code snipped I am using.
private Boolean useQueryWithQuotes=null;
private final String queryWithQuotes = "\"OWNER\"=? or \"PRIVATE\"=?";
private final String queryWithoutQuotes = "OWNER=? or PRIVATE=?";
public Response getReports() {
List<ReportEntity> reports = null;
if(useQueryWithQuotes==null){
synchronized(this){
try {
reports = new ArrayList<ReportEntity>( Arrays.asList(ao.find(ReportEntity.class, Query.select().where(queryWithoutQuotes, getUserKey(), false))) );
useQueryWithQuotes = false;
} catch (net.java.ao.ActiveObjectsException e) {
log("exception:" + e);
log("trying query with quotes");
reports = new ArrayList<ReportEntity>( Arrays.asList(ao.find(ReportEntity.class, queryWithQuotes, getUserKey(), false)));
useQueryWithQuotes = true;
}
}
}else{
String query = useQueryWithQuotes ? queryWithQuotes : queryWithoutQuotes;
reports = new ArrayList<ReportEntity>( Arrays.asList(ao.find(ReportEntity.class, query, getUserKey(), false)));
}
...
}
It is possible to preserve insertion order when parsing a JSON struct with a
Poco::JSON::Parser( new Poco::JSON::ParseHandler( true ) ): the non-default ParseHandler parameter preserveObjectOrder = true is handed over to the Poco::JSON::Objects so that they keep an private list of keys sorted in insertion order.
An object can then be serialized via Object::stringify() to look just like the source JSON string. Fine.
What, however, is the official way to step through a Poco::JSON::Object and access its internals in insertion order? Object::getNames() and begin()/end() use the alphabetical order of keys, not insertion order -- is there another way to access the values, or do I have to patch Poco?
As you already said:
Poco::JSON::ParseHandler goes into the Poco::JSON::Parser-constructor.
Poco::JSON::Parser::parse() creates a Poco::Dynamic::Var.
From that you'll extract a Poco::JSON::Object::Ptr.
The Poco::JSON:Object has the method "getNames". Beginning with this commit it seems to preserve the order, if it was requested via the ParseHandler. (Poco::JSON:Object::getNames 1.8.1, Poco::JSON:Object::getNames 1.9.0)
So now it should work as expected to use:
for(auto const & name : object->getNames()){
auto const & value = object->get(name); // or one of the other get-methods
// ... do things ...
}
I am trying to update data to a mySQL database using JPA. I have no problem persisting data but flush will not work as expected. I retrieve the id for the login session, set that id (it is the primary key) along with setting the description field that I want merged to the database. I have debugged line by line through this method and all variables contain the expected values. Any ideas or suggestions to overcome this problem are appreciated.
public String update() {
factory = Persistence.createEntityManagerFactory(PERSISTENCE_UNIT_NAME);
EntityManager em = factory.createEntityManager();
if(true){
em.getTransaction().begin();
String sessionEmail=Util.getEmail();
//Create query to find user passwords matching the inputted name
Query myQuery = em.createQuery("SELECT u FROM BusinessAccount u WHERE u.email=:email");
myQuery.setParameter("email", sessionEmail);
List<BusinessAccount> accounts=myQuery.getResultList();
int intId=accounts.get(0).getId();
businessAccount.setId(intId);
String des=businessAccount.getDescription();
businessAccount.setDescription(des);
em.flush();
addMessage(new FacesMessage(FacesMessage.SEVERITY_INFO,
"User Registration Successful!", null));
return "success";
}
else {
addMessage(new FacesMessage(FacesMessage.SEVERITY_ERROR,
"User Registration Failed!", null));
return "failure";
}
}
merge() persists all the state of the entity. Not just the non-null fields. I it wasn't, you would complain that you want to set some field to null and that merge() ignores it and leaves it as is.
So get an entity from the database, and modify it, instead of only gettings its ID, creating a new entity instance from scratch and only settings some of its fields.
Note that, if you get the entity and modify it inside a single transaction, you don't even have to call merge(): the new state will be made persistent automatically.
I would like to query a table based on a list of KeyValuePair. With a Model-First approach, I could do the following:
var context = new DataContext();
var whereClause = new StringBuilder();
var objectParameters = new List<ObjectParameter>();
foreach(KeyValuePair<string, object> pair in queryParameters)
{
if (whereClause.Length > 0)
whereClause.Append(" AND ");
whereClause.Append(string.Format("it.[{0}] = #{0}", pair.Key));
parameters.Add(new ObjectParameter(pair.Key, pair.Value));
}
var result = context.Nodes.Where(whereClause.ToString(), parameters.ToArray());
Now I'm using a Code-First approach and this Where method is not available anymore. Fortunately, I saw an article somewhere (I can't remember anymore) which suggested that I could convert the DbContext to a IObjectContextAdapter then call CreateQuery like this:
var result = ((IObjectContextAdapter)context)
.ObjectContext.CreateQuery<Node>(whereClause.ToString(), parameters.ToArray());
Unfortunately, this throws an error:
'{ColumnName}' could not be resolved in the current scope or context. Make sure that all referenced variables are in scope, that required schemas are loaded, and that namespaces are referenced correctly.
Where {ColumnName} is the column specified in the whereClause.
Any ideas how I can dynamically query a DbSet given a list of key/value pairs? All help will be greatly appreciated.
I think your very first problem is that in the first example you are using Where on the entity set but in the second example you are using CreateQuery so you must pass full ESQL query and not only where clause! Try something like:
...
.CreateQuery<Node>("SELECT VALUE it FROM ContextName.Nodes AS it WHERE " + yourWhere)
The most problematic is full entity set name in FROM part. I think it is defined as name of the context class and name of the DbSet exposed on the context. Another way to do it is creating ObjectSet:
...
.ObjectContext.CreateObjectSet<Node>().Where(yourWhere)