I'm trying to get a TermsResponse object from a solrj QueryResponse object, but it doesn't seem to be working. I'm using scala, but I would be happy with a working java example too.
First I set up the term vector query, which looks to be working:
val solrurl = "http://localhost:8983/solr"
val server= new HttpSolrServer( solrurl )
val query = new SolrQuery
query.setRequestHandler("/tvrh")
query.set("fl", "text")
query.set("tv.all", true)
query.setQuery("uid:" + id)
val response = server.query(query)
The query returns a QueryResponse object whose toString looks to be a JSON object. This object includes the term vector information (terms, frequency, etc . . .) as part of the JSON object.
But when I do this I always get a null object:
val termsResponse = Option(response.getTermsResponse)
Is this function deprecated?
If so what is the best way to retrieve the structure from QueryResponse? Convert to JSON? Some other sources point to using response.get("termVector") but that seems to be deprecated.
Any ideas?
Thanks
I have been using simple java object for this with the following configuration.
//Adding terms for 2 word phrases
qterms.setTerms(true);
qterms.setRequestHandler("/terms");
qterms.setTermsLimit(20);
qterms.addTermsField("PhraseIndx2");
qterms.setTermsMinCount(20);
QueryResponse response = solr.query(query);
SolrDocumentList results = response.getResults();
//queryresponse get all terms from in 2 phrase field
System.out.println ("printing the terms from queryresponse: \n");
QueryResponse resTerms = solr.query(qterms);
TermsResponse termResp =resTerms.getTermsResponse();
List<Term> terms = termResp.getTerms("PhraseIndx2");
System.out.print(terms.size());
Related
In MySQL, I have a field name postcodes and the type is LONGTEXT. It stores several postcodes being separated by comma. How would I retrieve that and store it as an array for other use ?
you can use the PHP method explode().
one think you can't do, is to do a where x = x on it in the database.
In the model, you can set the mutator methods:
public function getPostcodesAttribute($value) {
return explode(',',$value);
}
public function setPostcodesAttribute($value) {
$this->attributes['postcodes'] = implode(',',$value);
}
Lets say that you have the result stored in a string like this:
$s = "6000,5447"; //$s = $array->postcodes;
you can get the each value on an index in an array using this:
$values= explode(",", $s);
echo $values[0]; // 6000
Or even better.. you can store it as json, and retrieve it as json in array format.
Store it as a JSON field in MySQL, Laravel encode and decode them when you retrieve and save them respectively
in your migration
$table->json('field_name');
then in the model
protected $json = ['field_name'];
then whenever you access the field, laravel will convert it to an array for you, you don't have to call any explicit methods.
Doc - https://laravel.com/docs/5.7/eloquent-mutators#attribute-casting
// the final array all the post codes are collected.
$postCodes = [];
foreach (Model::pluck('postcodes') as $stringCodes)
foreach (explode(',', $stringCodes) as $postCode) $postCodes[] = $postCode;
Given there is a dataset of messages, defined by following code:
case class Message(id: Int, value: String)
var messages = Seq(
(0, """{"action":"update","timestamp":"2017-10-05T23:01:19Z"}"""),
(1, """{"action":"update","timestamp":"2017-10-05T23:01:19Z"}""")
).toDF("id", "value").as[Message]
var schema = new StructType().add("action", StringType).add("timestamp", TimestampType)
var res = messages.select(
from_json(col("value").cast("string"), schema)
)
+------------------------------------+
|jsontostructs(CAST(value AS STRING))|
+------------------------------------+
| [update,2017-10-0...|
| [update,2017-10-0...|
What is the best way to access the schema information in a plain map function. The function itself returns a row which has lost all the Type infos. In order to reach to the values one has to specify the type again e.g
res.head().getStruct(0).getValuesMap[TimestampType](Seq("timestamp"))
=> Map[String,org.apache.spark.sql.types.TimestampType] = Map(timestamp -> 2017-10-06 01:01:19.0)
or
res.head().getStruct(0).getString(0)
=> res20: String = update
Is there some better way to access the raw json data without spark sql aggregation functions?
As a rule of thumb:
To use collection API (map, flatMap, mapPartitions, groupByKey, etc.) use strongly typed API - define record type (case class works the best) which reflects the schema and use Encoders to convert things back and forth:
case class Value(action: String, timestamp: java.sql.Timestamp)
case class ParsedMessage(id: Int, value: Option[Value])
messages.select(
$"id", from_json(col("value").cast("string"), schema).alias("value")
).as[ParsedMessage].map(???)
With Dataset[Row] stay with high level SQL / DataFrame API (select, where, agg, groupBy)
is there anyway to make a query in MongoDB using a JSON and returning a object if one field of the json matches with some in the database?
for example, I have the this object called keysArray
{ house: 'true', garden: 'false' }
and I would like to make a query in Mongo passing this object as a query field and return if some object in my database matches with at least one of those fields :
keysArray.forEach(function(key){
collection.find({keysArray}, function(err, propertyMatch){
console.log(propertyMatch)
})
})
I got no objects back, even if I have one object in my database that matches these fields.
Thanks in advance
...and I would like to make a query in Mongo passing this object as a
query field and return if some object in my database matches with at
least one of those fields.
It sounds like OR logic - if I understood it well.
On this specific case it's not possible to pass in JSON-like object to query as it would be a implicit AND logic condition.
So you should build first a OR expression and use it in collection.find(), something like this:
var myjson = {'status': 32, 'profile': {$exists: false}};
function build_logic_or(json) {
var orExpr = [];
for (var field in json) {
var expr = {};
expr[field] = json[field];
orExpr.push(expr);
}
return {'$or': orExpr};
}
It would build an expression like this:
{"$or":[{"status":32},{"profile":{"$exists":false}}]}
So:
db.collection.find(build_logic_or(myjson))
This code seems to work fine when I used Django console and just print it.
reference = FloodHazard.objects.filter(hazard='High')
ids = reference.values_list('id', flat=True)
for myid in ids:
getgeom = FloodHazard.objects.get(id=myid).geom
response = BuildingStructure.objects.filter(geom__intersects=getgeom).values(
'brgy_locat').annotate(counthigh=Count('brgy_locat'))
print response
I was able to show all the values, but when using HttpResponse, it returns an empty set. What is the proper way of returning JSON data from a queryset? So far, tried this:
reference = FloodHazard.objects.filter(hazard='High')
ids = reference.values_list('id', flat=True)
response = {}
for myid in ids:
getgeom = FloodHazard.objects.get(id=myid).geom
response['high'] = BuildingStructure.objects.filter(geom__intersects=getgeom).values(
'brgy_locat').annotate(counthigh=Count('brgy_locat'))
json_post = ujson.dumps(list(response))
return HttpResponse(json_post, content_type='application/json')
There is no much sense in your code. You assign all querysets to the single key in the response dict. You should use a list for this purpose:
As far as I understand the code should be something like this:
response = []
for myid in ids:
getgeom = FloodHazard.objects.get(id=myid).geom
response.extend(BuildingStructure.objects.filter(geom__intersects=getgeom)
.values('brgy_locat')
.annotate(counthigh=Count('brgy_locat')))
json_post = ujson.dumps(response)
If you want to return a hazard level as well as the list of buildings then you can return a dict:
json_post = ujson.dumps({'hazard': 'high', 'buildings': response})
I'm using the Jackson Object Mapper (com.fasterxml.jackson.databind.ObjectMapper) to parse a json string and need to count the number of times the word "path"
occures in the string. The string looks like:
"rows":[{"path":{"uid":"2"},"fields":[]},{"path":{"uid":"4"},"fields":[]},{"path":{"uid":"12"},....
Does anyone know which API option is most efficeint for achieving this?
To count total number of 'childs' inside the 'rows' root, you can use a code like this:
String inputJsonString = "{\"rows\":[{\"path\":{\"uid\":\"2\"},\"fields\":[]},{\"path\":{\"uid\":\"4\"},\"fields\":[]},{\"path\":{\"uid\":\"12\"},\"fields\":[]}]}";
ObjectNode root = (ObjectNode) new ObjectMapper().readTree( inputJsonString );
root.get( "rows" ).size();
If you need to get the exact count of 'path' occurrences, you may use the code like this:
int counter = 0;
for( Iterator<JsonNode> i = root.get( "rows" ).iterator(); i.hasNext(); )
if( i.next().has( "path" ) )
counter++;
System.out.println(counter);