I have defined a #Highlight query on my Solr repository class which returns the interesting 'snipplets' info if I call it from another class/controller. However if I access the method through the implicit rest service exposed via #RepositoryRestResource ( e.g. http://localhost:8080/modules/search/findByDescriptionContaining?description=maths ) , it doesn't return any highlighting/snipplet info at all
#RepositoryRestResource
interface ModuleRepository extends SolrCrudRepository<Module, String>{
#Highlight(prefix = "<b>", postfix = "</b>")
HighlightPage<Module> findByDescriptionContaining(#Param(value = "description") String description, Pageable pageable)
}
so to recap, calling the auto exposed method will return this:
{
"_embedded" : {
"modules" : [ {
"name" : "science",
"description" : "a somewhat scientific course with some maths",
"_links" : {
"self" : {
"href" : "http://localhost:8080/modules/102"
},
"module" : {
"href" : "http://localhost:8080/modules/102"
}
}
} ]
},
"_links" : {
"self" : {
"href" : "http://localhost:8080/modules/search/findByDescriptionContaining?description=maths"
}
},
"page" : {
"size" : 20,
"totalElements" : 1,
"totalPages" : 1,
"number" : 0
}
}
how can I get the highlighting info using the method above ?
Related
I encountered a problem which is very challenging to my Angular level. Could you give a help please?
In Spring Data REST the entity Worker has a #OneToMany bidirectional relationship with the entity TempworkEvent, shown below under _links. I would like to access TempworkEvent objects and their attributes through this relationship.
{
"id" : 3,
"name" : "Nadja Miller",
"profession" : "Experte/in Anästhesiepflege",
"_links" : {
"self" : {
"href" : "http://localhost:8080/api/workers/3"
},
"worker" : {
"href" : "http://localhost:8080/api/workers/3"
},
"tempworks" : {
"href" : "http://localhost:8080/api/workers/3/tempworks"
},
"tempworkEvents" : {
"href" : "http://localhost:8080/api/workers/3/tempworkEvents"
}
}
In Angular the Worker entity successfully:
getWorkers(): Observable<Worker[]> {
return this.httpClient.get<GetResponseWorkers>(this.workersUrl).pipe(
map(response => response._embedded.workers)
);
}
(...)
interface GetResponseWorkers {
_embedded: {
workers: Worker[];
}
}
The challenge is how to access the Worker's tempworkEvents object and its attributes through _links:
"tempworkEvents" : {
"href" : "http://localhost:8080/api/workers/3/tempworkEvents"
}
One of:
Make an additional network request to get the TempworkEvents
Don't have a TempworkEvent JPA repository, and Spring Data Rest will include the tempworkEvents data in the workers response
or, use a Projection, like:
in entities/projections/MyWorkerProjection.java:
#Projection(name="foo", types={Worker.class})
public interface MyWorkerProjection {
Long getId();
String getName();
String getProfession();
List<TempworkEvent> getTempworkEvents();
}
Then call http://localhost:8080/api/workers/3?projection=foo
If you want to have "auto" projections, see: Spring Data REST: projection representation of single resource
You may want to create another interface for TempworkEvent, if you want to project its data as well as its links
let obj = {
"id" : 3,
"name" : "Nadja Miller",
"profession" : "Experte/in Anästhesiepflege",
"_links" : {
"self" : {
"href" : "http://localhost:8080/api/workers/3"
},
"worker" : {
"href" : "http://localhost:8080/api/workers/3"
},
"tempworks" : {
"href" : "http://localhost:8080/api/workers/3/tempworks"
},
"tempworkEvents" : {
"href" : "http://localhost:8080/api/workers/3/tempworkEvents"
}
}
}
console.log('tempworkEvents',obj._links.tempworkEvents);
You can access the object like above.
I am getting a JSON response and I want to display the contents obtained from this response on my HTML page using angular.
ngOnInit() {
this.http.get(this.url)
.subscribe(response =>{
this.users = response.json();
});
}
Above code returns an Object of JSON which cause an error if iterated using ngFor.
I have tried some answers available on StackOverflow regarding this problem but they cause compilation errors or give no results.
Here is JSON Object I'm getting
{
"_embedded" : {
"users" : [ {
"name" : "Ahmad",
"role" : "Admin",
"_links" : {
"self" : {
"href" : "http://localhost:8080/users/1"
},
"user" : {
"href" : "http://localhost:8080/users/1"
}
}
}, {
"name" : "Umar",
"role" : "User",
"_links" : {
"self" : {
"href" : "http://localhost:8080/users/2"
},
"user" : {
"href" : "http://localhost:8080/users/2"
}
}
}, {
"name" : "Ali",
"role" : "Admin",
"_links" : {
"self" : {
"href" : "http://localhost:8080/users/3"
},
"user" : {
"href" : "http://localhost:8080/users/3"
}
}
}, {
"name" : "Waqas",
"role" : "User",
"_links" : {
"self" : {
"href" : "http://localhost:8080/users/4"
},
"user" : {
"href" : "http://localhost:8080/users/4"
}
}
} ]
},
"_links" : {
"self" : {
"href" : "http://localhost:8080/users{?page,size,sort}",
"templated" : true
},
"profile" : {
"href" : "http://localhost:8080/profile/users"
},
"search" : {
"href" : "http://localhost:8080/users/search"
}
},
"page" : {
"size" : 20,
"totalElements" : 4,
"totalPages" : 1,
"number" : 0
}
}
EDIT: HTML code with ngFor
<ul class="list-group">
<li
*ngFor="let user of users"
class="list-group-item">
{{ user }}
</li>
I am learning angular at the moment so please give some explanation with the answer.
You need to modify this
this.users = response.json();
to
this.users=response.json()._embedded.users
You need to modify this
this.users = response.json();
to
this.users = response.json()['_embedded']['users'];
I'm trying to convert JSON into Avro using the kite-sdk morphline module. After playing around I'm able to convert the JSON into Avro using a simple schema (no complex data types).
Then I took it one step further and modified the Avro schema as displayed below (subrec.avsc). As you can see the schema consist of a subrecord.
As soon as I tried to convert the JSON to Avro using the morphlines.conf and the subrec.avsc it failed.
Somehow the JSON paths "/record_type[]/alert/action" are not translated by the toAvro function.
The morphlines.conf
morphlines : [
{
id : morphline1
importCommands : ["org.kitesdk.**"]
commands : [
# Read the JSON blob
{ readJson: {} }
{ logError { format : "record: {}", args : ["#{}"] } }
# Extract JSON
{ extractJsonPaths { flatten: false, paths: {
"/record_type[]/alert/action" : /alert/action,
"/record_type[]/alert/signature_id" : /alert/signature_id,
"/record_type[]/alert/signature" : /alert/signature,
"/record_type[]/alert/category" : /alert/category,
"/record_type[]/alert/severity" : /alert/severity
} } }
{ logError { format : "EXTRACTED THIS : {}", args : ["#{}"] } }
{ extractJsonPaths { flatten: false, paths: {
timestamp : /timestamp,
event_type : /event_type,
source_ip : /src_ip,
source_port : /src_port,
destination_ip : /dest_ip,
destination_port : /dest_port,
protocol : /proto,
} } }
# Create Avro according to schema
{ logError { format : "WE GO TO AVRO"} }
{ toAvro { schemaFile : /etc/flume/conf/conf.empty/subrec.avsc } }
# Create Avro container
{ logError { format : "WE GO TO BINARY"} }
{ writeAvroToByteArray { format: containerlessBinary } }
{ logError { format : "DONE!!!"} }
]
}
]
And the subrec.avsc
{
"type" : "record",
"name" : "Event",
"fields" : [ {
"name" : "timestamp",
"type" : "string"
}, {
"name" : "event_type",
"type" : "string"
}, {
"name" : "source_ip",
"type" : "string"
}, {
"name" : "source_port",
"type" : "int"
}, {
"name" : "destination_ip",
"type" : "string"
}, {
"name" : "destination_port",
"type" : "int"
}, {
"name" : "protocol",
"type" : "string"
}, {
"name": "record_type",
"type" : ["null", {
"name" : "alert",
"type" : "record",
"fields" : [ {
"name" : "action",
"type" : "string"
}, {
"name" : "signature_id",
"type" : "int"
}, {
"name" : "signature",
"type" : "string"
}, {
"name" : "category",
"type" : "string"
}, {
"name" : "severity",
"type" : "int"
}
] } ]
} ]
}
The output on { logError { format : "EXTRACTED THIS : {}", args : ["#{}"] } } I output the following:
[{
/record_type[]/alert / action = [allowed],
/record_type[]/alert / category = [],
/record_type[]/alert / severity = [3],
/record_type[]/alert / signature = [GeoIP from NL,
Netherlands],
/record_type[]/alert / signature_id = [88006],
_attachment_body = [{
"timestamp": "2015-03-23T07:42:01.303046",
"event_type": "alert",
"src_ip": "1.1.1.1",
"src_port": 18192,
"dest_ip": "46.231.41.166",
"dest_port": 62004,
"proto": "TCP",
"alert": {
"action": "allowed",
"gid": "1",
"signature_id": "88006",
"rev": "1",
"signature" : "GeoIP from NL, Netherlands ",
"category" : ""
"severity" : "3"
}
}],
_attachment_mimetype=[json/java + memory],
basename = [simple_eve.json]
}]
UPDATE 2017-06-22
you MUST populate the data in the structure in order for this to work, by using addValues or setValues
{
addValues {
micDefaultHeader : [
{
eventTimestampString : "2017-06-22 18:18:36"
}
]
}
}
after debugging the sources of morphline toAvro, it appears that the record is the first object to be evaluated, no matter what you put in your mappings structure.
the solution is quite simple, but unfortunately took a little extra time, eclipse, running the flume agent in debug mode, cloning the source code and lots of coffee.
here it goes.
my schema:
{
"type" : "record",
"name" : "co_lowbalance_event",
"namespace" : "co.tigo.billing.cboss.lowBalance",
"fields" : [ {
"name" : "dummyValue",
"type" : "string",
"default" : "dummy"
}, {
"name" : "micDefaultHeader",
"type" : {
"type" : "record",
"name" : "mic_default_header_v_1_0",
"namespace" : "com.millicom.schemas.root.struct",
"doc" : "standard millicom header definition",
"fields" : [ {
"name" : "eventTimestampString",
"type" : "string",
"default" : "12345678910"
} ]
}
} ]
}
morphlines file:
morphlines : [
{
id : convertJsonToAvro
importCommands : ["org.kitesdk.**"]
commands : [
{
readJson {
outputClass : java.util.Map
}
}
{
addValues {
micDefaultHeader : [{}]
}
}
{
logDebug { format : "my record: {}", args : ["#{}"] }
}
{
toAvro {
schemaFile : /home/asarubbi/Development/test/co_lowbalance_event.avsc
mappings : {
"micDefaultHeader" : micDefaultHeader
"micDefaultHeader/eventTimestampString" : eventTimestampString
}
}
}
{
writeAvroToByteArray {
format : containerlessJSON
codec : null
}
}
]
}
]
the magic lies here:
{
addValues {
micDefaultHeader : [{}]
}
}
and in the mappings:
mappings : {
"micDefaultHeader" : micDefaultHeader
"micDefaultHeader/eventTimestampString" : eventTimestampString
}
explanation:
inside the code the first field name that is evaluated is micDefaultHeader of type RECORD. as there's no way to specify a default value for a RECORD (logically correct), the toAvro code evaluates this, does not get any value configured in mappings and therefore it fails at it detects (wrongly) that the record is empty when it shouldn't.
however, taking a look at the code, you may see that it requires a Map object, containing no values to please the parser and continue to the next element.
so we add a map object using the addValues and fill it with an empty map [{}]. notice that this must match the name of the record that is causing you an empty value. in my case "micDefaultHeader"
feel free to comment if you have a better solution, as this looks like a "dirty fix"
I would like to pluck documents from a JSON payload and send them to different processors based on their location.
JSON:
{
"email" : {
"documents" : [{
"name" : "Document 1",
"id" : "1111"
}, {
"name" : "Document 2",
"id" : "222"
}
]
},
"sms" : {
"documents" : [{
"name" : "Document 3",
"id" : "3333"
}, {
"name" : "Document 4",
"id" : "4444"
}
]
}
}
I was thinking to achieve this by doing something like this:
from("servlet:///doc").unmarshal()
.json(JsonLibrary.Jackson, DocumentRequest.class)
.split().method("docSplit", "split")
.choice()
.when().header("mode").isEqualTo("email")
.to("direct:email")
.when().header("mode").isEqualTo("sms")
.to("direct:sms");
My splitter can receive a DocumentRequest and pull out all the docs... but I do not know how to set the "mode" header for future routing.
How can I set the "mode" header?
Is there a better overall approach?
You can always put custom split logic in a custom processor and use ProducerTemplate
For example:
from("servlet:///doc").unmarshal()
.json(JsonLibrary.Jackson, DocumentRequest.class)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
ProducerTemplate producer=exchange.getContext().createProducerTemplate();
String mode;
for (Document doc: // split and set mode logic goes here ) {
if (mode.compareToIgnoreCase("email") ==0) {
producer.sendBody("direct:email", doc);
} else
if (mode.compareToIgnoreCase("sms") ==0) {
producer.sendBody("direct:sms", doc);
}
...
}
}
});
I'm looking for a best way to design a custom JSON response for RESTful API in Grails based on this presentation Design Beautiful REST + JSON APIs by Les Hazlewood.
Here is my Domain class
class TaxiType {
Date dateCreated, lastUpdated
String description
User createdBy
static hasMany = [taxis: Taxi]
static constraints = {
}
}
The desired response format for list is
{
"meta": {
"href": "https://api.mydomain.com/taxi-types",
"otherData": "..."
},
"paging": {
"offset": 0,
"limit": 10,
"total": 100,
"first": "https://api.mydomain.com/taxi-types?offset=0&limit=10",
"previous": null,
"next": "https://api.mydomain.com/taxi-types?offset=90&limit=10",
"last": "https://api.mydomain.com/taxi-types?offset=90&limit=10"
},
"data": [
{
"href": "https://api.mydomain.com/taxi-types/1",
"id": 1,
"description": "description 1",
"taxis": {
"href": "https://api.mydomain.com/taxi-types/1/taxis"
}
},
...
]
}
TaxiTypeController.groovy
def index(Integer limit) {
params.max = Math.min(limit ? : 10, 100)
params.offset = params ? .offset ? .toInteger()
withFormat {
json {
respond TaxiType.list(params),
[includes : includeFields,
paging : [total : TaxiType.count(), limit : params ? .max, offset : params ? .offset ? : 0]
]
}
}
}
private getIncludeFields() {
params.fields ? .tokenize(', ')
}
SumoJsonCollectionRenderer.groovy
class SumoJsonCollectionRenderer extends JsonCollectionRenderer {
SumoJsonCollectionRenderer(Class componentType) {
super(componentType)
}
public SumoJsonCollectionRenderer(Class componentType, MimeType...mimeTypes) {
super(componentType, mimeTypes)
}
# CompileStatic(SKIP)
# Override
protected void renderJson(object, RenderContext context) {
log.debug(object)
log.debug(object.size())
log.debug(object.getTotalCount())
Map tObject = ['data' : object]
if (context.arguments ? .paging) {
tObject['paging'] = context ? .arguments ? .paging
}
super.renderJson(tObject, context)
}
}
Required features are:
1) API users should be able to get only the required fields (Partial representations)
GET https://api.mydomain.com/taxi-types??fields=id,description,taxis
the desired response for this request should be
{
"meta" : {...}
"paging" : {...}
"data" : [{
"href" : "https://api.mydomain.com/taxi-types/1",
"id" : 1,
"description" : "Taxi Type1",
"taxis" : [{
"href" : "https://api.mydomain.com/taxis/123"
},
...
]
},
...
]
}
What I'm getting was
{
"data" : [{
"id" : 1,
"description" : "Taxi Type1",
"taxis" : [{
"class" : "com.domain.project.Taxi",
"id" : 1
}
]
},
...
],
"paging" : {
"total" : 80,
"limit" : 10,
"offset" : 0
}
}
I referred answer to this question Render metadata for pagination from Grails RestfulController index/search actions.
But still need to include links for first, previous, next & last in paging as mentioned in the desired format above.
2) Customizing output
for example
the taxis property with hasMany relationship should be rendered by default as
"taxis": {
"href": "https://api.mydomain.com/taxis/12345"
}
and if the user prefer to expand taxis property, for eg: GET /taxi-types?expand=taxis, the JSON format should be
"taxis": {
"href": "https://api.mydomain.com/taxis/12345"
"name": "Taxi name",
"type": "https://api.mydomain.com/taxi-types/1"
...
}
3) Add meta object to all responses
"meta": {
"href": "https://api.mydomain.com/taxi-types",
...
}
I tried Json Marshaller to customize the response. Here is my Marshaller code
JSON.registerObjectMarshaller(TaxiType) {TaxiType taxiType->
return [
id : taxiType ? .id,
description : taxiType ? .description
]
}
but it always renders with those two properties in return array id & description, even if I want some other properties like taxis.
How above 3 requirements can be implemented?. Thanks in advance.
I've created a plugin for grails 3 which allows you to add pagination, custom metadata and other useful features to your apis. You can add HAL features using LinkGenerators as well with my plugin.
well the Beapi-API-Framework plugin is the #1 api plugin for grails and provides pagination by default. See the github repo for a complete list of all functionality, documentation and install instructions