Can we create the object of AwsCrypto just once and reuse it for all the request? Basically I want to create the KMSClient with AwsCrypto which can be used for encryption and decryption for multiple CMKs.
Below is the sample code:
public class KMSClient {
final private AwsCrypto awsCrypto;
final private Map<String, KmsMasterKeyProvider> kmsMasterKeyProviderMap;
public KMSClient(AwsCrypto awsCrypto){
this.awsCrypto = awsCrypto;
this.kmsMasterKeyProviderMap = new HashMap<>();
}
public byte[] encrypt(final byte[] data, final String customerManagedKeyId) {
return awsCrypto.encryptData(getKmsMasterKeyProvider(customerManagedKeyId), data).getResult();
}
public byte[] decrypt(final byte[] data, final String customerManagedKeyId) {
return awsCrypto.decryptData(getKmsMasterKeyProvider(customerManagedKeyId), data).getResult();
}
private KmsMasterKeyProvider getKmsMasterKeyProvider(final String customerManagedKeyId){
return kmsMasterKeyProviderMap.computeIfAbsent(customerManagedKeyId,
k -> KmsMasterKeyProvider.builder().withKeysForEncryption(k).build());
}
AwsCrypto is thread safe except for setEncryptionAlgorithm and setEncryptionFrameSize. Those two methods are safe to call from multiple threads as long as every thread using that object is on board with changing the state. If one thread makes a change without others being ready for it, there will be trouble.
Related
I am trying to receive and access JSON data from a Kafka Topic in Flink. What works is, producing data, send it to a Kafka Topic und receive it in Flink as String. But I want to access the data in an object-oriented way (e.g. extract a specific atrribute from every message)?
Therefore I have a Kafka Producer which sends data (e.g. every 1s) to a Kafka Topic:
ObjectMapper test = new ObjectMapper();
ObjectNode jNode= test.createObjectNode();
jNode.put("LoPos", longPos)
.put("LaPos", latPos)
.put("Timestamp", timestamp.toString());
ProducerRecord<String, ObjectNode> rec = new ProducerRecord<String, ObjectNode>(topicName, jNode);
producer.send(rec);
so the JSON data looks like this:
{"LoPos":10.5,"LaPos":2.5,"Timestamp":"2022-10-31 12:45:19.353"}
What works is, receiving the data and print it as string:
DataStream<String> input =
env.fromSource(
KafkaSource.<String>builder()
.setBootstrapServers("localhost:9092")
.setBounded(OffsetsInitializer.latest())
.setValueOnlyDeserializer(new SimpleStringSchema())
.setTopics(topicName)
.build(),
WatermarkStrategy.noWatermarks(),
"kafka-source");
Print the data as string:
DataStream<String> parsed = input.map(new MapFunction<String, String>() {
private static final long serialVersionUID = -6867736771747690202L;
#Override
public String map(String value) {
System.out.println(value);
return "test";
How can I receive the data in Flink and access it in an object-oriented way (e.g. extract LoPos from every message)? Which approach would you recommend? I tried it with JSONValueDeserializationSchema, but without success...
Thanks!
Update1:
I updated to Flink 1.16 to use JsonDeserializationSchema.
Then I created a Flink Pojo Event like this:
public class Event {
public double LoPos;
public double LaPos;
public Timestamp timestamp;
public Event() {}
public Event(final double LoPos, final double LaPos, final Timestamp timestamp) {
this.LaPos=LaPos;
this.LoPos=LoPos;
this.timestamp=timestamp;
}
#Override
public String toString() {
return String.valueOf(LaPos);
}
}
To read the JSON data, I implemented the following:
KafkaSource<Event> source = KafkaSource.<Event>builder()
.setBootstrapServers("localhost:9092")
.setBounded(OffsetsInitializer.earliest())
.setValueOnlyDeserializer(new JsonDeserializationSchema<>(Event.class))
.setTopics("testTopic2")
.build();
DataStream<Event> test=env.fromSource(source, WatermarkStrategy.noWatermarks(), "test");
System.out.println(source.toString());
System.out.println(test.toString());
//test.sinkTo(new PrintSink<>());
test.print();
env.execute();
So I would expect, when using source.toString() the value of LaPos is getting returned. But all I get is:
org.apache.flink.connector.kafka.source.KafkaSource#510f3d34
What am I doing wrong?
This topic is covered in one of the recipes in the Immerok Apache Flink Cookbook.
In the examples below, I'm assuming Event is a Flink POJO.
With Flink 1.15 or earlier, you should use a custom deserializer:
KafkaSource<Event> source =
KafkaSource.<Event>builder()
.setBootstrapServers("localhost:9092")
.setTopics(TOPIC)
.setStartingOffsets(OffsetsInitializer.earliest())
.setValueOnlyDeserializer(new EventDeserializationSchema())
.build();
The deserializer can be something like this:
public class EventDeserializationSchema extends AbstractDeserializationSchema<Event> {
private static final long serialVersionUID = 1L;
private transient ObjectMapper objectMapper;
/**
* For performance reasons it's better to create on ObjectMapper in this open method rather than
* creating a new ObjectMapper for every record.
*/
#Override
public void open(InitializationContext context) {
// JavaTimeModule is needed for Java 8 data time (Instant) support
objectMapper = JsonMapper.builder().build().registerModule(new JavaTimeModule());
}
/**
* If our deserialize method needed access to the information in the Kafka headers of a
* KafkaConsumerRecord, we would have implemented a KafkaRecordDeserializationSchema instead of
* extending AbstractDeserializationSchema.
*/
#Override
public Event deserialize(byte[] message) throws IOException {
return objectMapper.readValue(message, Event.class);
}
}
We've made this easier in Flink 1.16, where we've added a proper JsonDeserializationSchema you can use:
KafkaSource<Event> source =
KafkaSource.<Event>builder()
.setBootstrapServers("localhost:9092")
.setTopics(TOPIC)
.setStartingOffsets(OffsetsInitializer.earliest())
.setValueOnlyDeserializer(new JsonDeserializationSchema<>(Event.class))
.build();
Disclaimer: I work for Immerok.
In my spring boot service, I'm using https://github.com/java-json-tools/json-patch for handling PATCH requests.
Everything seems to be ok except a way to avoid modifying immutable fields like object id's, creation_time etc. I have found a similar question on Github https://github.com/java-json-tools/json-patch/issues/21 for which I could not find the right example.
This blog seems to give some interesting solutions about validating JSON patch requests with a solution in node.js. Would be good to know if something similar in JAVA is already there.
Under many circumstances you can just patch an intermediate object which only has fields that the user can write to. After that you could quite easily map the intermediate object to your entity, using some object mapper or just manually.
The downside of this is that if you have a requirement that fields must be explicitly nullable, you won’t know if the patch object set a field to null explicitly or if it was never present in the patch.
What you can do too is abuse Optionals for this, e.g.
public class ProjectPatchDTO {
private Optional<#NotBlank String> name;
private Optional<String> description;
}
Although Optionals were not intended to be used like this, it's the most straightforward way to implement patch operations while maintaining a typed input. When the optional field is null, it was never passed from the client. When the optional is not present, that means the client has set the value to null.
Instead of receiving a JsonPatch directly from the client, define a DTO to handle the validation and then you will later convert the DTO instance to a JsonPatch.
Say you want to update a user of instance User.class, you can define a DTO such as:
public class UserDTO {
#Email(message = "The provided email is invalid")
private String username;
#Size(min = 2, max = 10, message = "firstname should have at least 2 and a maximum of 10 characters")
private String firstName;
#Size(min = 2, max = 10, message = "firstname should have at least 2 and a maximum of 10 characters")
private String lastName;
#Override
public String toString() {
return new Gson().toJson(this);
}
//getters and setters
}
The custom toString method ensures that fields that are not included in the update request are not prefilled with null values.
Your PATCH request can be as follows(For simplicity, I didn't cater for Exceptions)
#PatchMapping("/{id}")
ResponseEntity<Object> updateUser(#RequestBody #Valid UserDTO request,
#PathVariable String id) throws ParseException, IOException, JsonPatchException {
User oldUser = userRepository.findById(id);
String detailsToUpdate = request.toString();
User newUser = applyPatchToUser(detailsToUpdate, oldUser);
userRepository.save(newUser);
return userService.updateUser(request, id);
}
The following method returns the patched User which is updated above in the controller.
private User applyPatchToUser(String detailsToUpdate, User oldUser) throws IOException, JsonPatchException {
ObjectMapper objectMapper = new ObjectMapper();
// Parse the patch to JsonNode
JsonNode patchNode = objectMapper.readTree(detailsToUpdate);
// Create the patch
JsonMergePatch patch = JsonMergePatch.fromJson(patchNode);
// Convert the original object to JsonNode
JsonNode originalObjNode = objectMapper.valueToTree(oldUser);
// Apply the patch
TreeNode patchedObjNode = patch.apply(originalObjNode);
// Convert the patched node to an updated obj
return objectMapper.treeToValue(patchedObjNode, User.class);
}
Another solution would be to imperatively deserialize and validate the request body.
So your example DTO might look like this:
public class CatDto {
#NotBlank
private String name;
#Min(0)
#Max(100)
private int laziness;
#Max(3)
private int purringVolume;
}
And your controller can be something like this:
#RestController
#RequestMapping("/api/cats")
#io.swagger.v3.oas.annotations.parameters.RequestBody(
content = #Content(schema = #Schema(implementation = CatDto.class)))
// ^^ this passes your CatDto model to swagger (you must use springdoc to get it to work!)
public class CatController {
#Autowired
SmartValidator validator; // we'll use this to validate our request
#PatchMapping(path = "/{id}", consumes = "application/json")
public ResponseEntity<String> updateCat(
#PathVariable String id,
#RequestBody Map<String, Object> body
// ^^ no Valid annotation, no declarative DTO binding here!
) throws MethodArgumentNotValidException {
CatDto catDto = new CatDto();
WebDataBinder binder = new WebDataBinder(catDto);
BindingResult bindingResult = binder.getBindingResult();
binder.bind(new MutablePropertyValues(body));
// ^^ imperatively bind to DTO
body.forEach((k, v) -> validator.validateValue(CatDto.class, k, v, bindingResult));
// ^^ imperatively validate user input
if (bindingResult.hasErrors()) {
throw new MethodArgumentNotValidException(null, bindingResult);
// ^^ this can be handled by your regular exception handler
}
// Here you can do normal stuff with your cat DTO.
// Map it to cat model, send to cat service, whatever.
return ResponseEntity.ok("cat updated");
}
}
No need for Optional's, no extra dependencies, your normal validation just works, your swagger looks good. The only problem is, you don't get proper merge patch on nested objects, but in many use cases that's not even required.
I need to insert a data multiple times in an web application. I am using selenium with testNG along with data driven framework.
I am using CSV file for reading the the input values.
Please find the sample code below.
public class TestData
{
private static String firstName;
public static String lastName;
#BeforeClass
public void beforeClass() throws IOException
{
reader = new CSVReader(new FileReader(fileName));
while((record = reader.readNext()) != null)
{
firstName = record[0];
lastName = record[1];
}
}
#Test
public void test1()
{
driver.findElement(By.id(id)).sendKeys(firstName);
driver.findElement(By.id(id)).click();
and so on....
}
#Test
public void test2()
{
driver.findElement(By.id(id)).sendKeys(lastName);
driver.findElement(By.id(id)).click();
and so on....
}
}
Here, I need to insert 3 records, but when I use the above code, only the 3rd record gets inserted.
Kindly help me to fix this issue.
Sample Input File
What you need here is a Factory powered by a DataProvider. The Factory would produce test class instances (A test class here is basically a regular class that contains one or more #Test methods housed in it). The data provider would basically feed the factory method with the data required to instantiate the test class.
Now your #Test methods would basically work with the data members in the instances to run its logic.
Here's a simple sample that shows this in action.
import org.assertj.core.api.Assertions;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;
public class TestClassSample {
private String firstName;
private String lastName;
#Factory(dataProvider = "dp")
public TestClassSample(String firstName, String lastName) {
this.firstName = firstName;
this.lastName = lastName;
}
#DataProvider(name = "dp")
public static Object[][] getData() {
//feel free to replace this with the logic that reads up a csv file (using CSVReader)
// and then translates it to a 2D array.
return new Object[][]{
{"Mohan", "Kumar"},
{"Kane", "Williams"},
{"Mark", "Henry"}
};
}
#Test
public void test1() {
Assertions.assertThat(this.firstName).isNotEmpty();
}
#Test
public void test2() {
Assertions.assertThat(this.lastName).isNotEmpty();
}
}
As per the data given by you , the while loop ends at the third record of CSV file. In each iteration your variables "firstName" and "lastName" are overwritten.
When the loop breaks , the variables store the lastly written values. So , use a better data structure for storing all values. I recommend map.
You can further club all the test cases in a single method , use invocationcount attribute in #Test annotation to repeat the execution for each entry from map. Add one more method with #BeforeTest for increment to next keyset in map.
I'm using Postman to test REST Web API and I have a method that receives this object:
public class NewShipmentDto implements Serializable {
private String description;
private Long senderId;
private Long receiverId;
private byte[] packageImage;
private CommissionPaidBy commissionPaidBy;
private Long senderPaymentMethod;
private LocationDto senderLocation;
private LocationDto receiverLocation;
// + constructors and gets/sets
}
So the body of my POST needs to be something like this:
{
"description":"Libros",
"senderId":1,
"receiverId":1,
"commissionPaidBy":"BOTH",
"senderPaymentMethod":1,
"senderLocation":
{
"latitud":100,
"longitud":100
},
"receiverLocation":
{
"latitud":100,
"longitud":100
}
}
But I also need to send the file so it can be transformed to a byte []. The thing is if I use form data to pass the file how do I send this bits:
"receiverLocation":
{
"latitud":100,
"longitud":100
}
Is it possible to send nested json in the form data?
This is my POST method, I haven't added the logic of reading the file yet:
#POST
#Consumes({MediaType.APPLICATION_JSON})
#Produces({MediaType.APPLICATION_JSON})
public Response addShipment(String shipment) throws EnviosYaException {
Gson gson = new Gson();
NewShipmentDto newShipmentDto = gson.fromJson(shipment, NewShipmentDto.class);
NewShipmentResponseDto response = shipmentServices.addShipment(newShipmentDto);
return Response.ok(gson.toJson(response)).build();
}
If necessary I could change the structure of the json to avoid having nested json, but I would like to know if it's possible first.
Thanks
It looks like you are using JAX-RS for the REST API. For your use case multipart/format-data content-type is more appropriate.
Please have a look at this JAX-RS question.
Or
If you like keep the JSON structure intact, you can store the base64 encoded data in a string for the image field. The model would looks like.
public class NewShipmentDto implements Serializable {
private String description;
private Long senderId;
private Long receiverId;
private String encodedPackageImage;
private CommissionPaidBy commissionPaidBy;
private Long senderPaymentMethod;
private LocationDto senderLocation;
private LocationDto receiverLocation;
// + constructors and gets/sets
}
First off my question is very similar to below however I'm not sure if the answers are applicable to my specific problem or whether I just need clarification about how to approach it:
Convert LinkedHashMap<String,String> to an object in Java
I am using struts2 json rest plugin to convert a json array into a java array. The array is sent through an ajax post request and the java receives this data. However instead of being the object type I expect it is received as a LinkedHashmap. Which is identical to the json request in structure.
[
{advance_Or_Premium=10000, available=true},
{advance_Or_Premium=10000, available=true},
{advance_Or_Premium=10000, available=true}
]
The data is all present and correct but just in the wrong type. Ideally I want to send the data in my object type or if this is not possible convert the LinkedHashMap from a list of keys and values into the object array. Here is the class I am using, incoming data is received in the create() method:
#Namespace(value = "/rest")
public class OptionRequestAction extends MadeAbstractAction implements ModelDriven<ArrayList<OptionRequestRest>>
{
private String id;
ArrayList<OptionRequestRest> model = new ArrayList<OptionRequestRest>();
public HttpHeaders create()
{
// TODO - need to use model here but it's a LinkedHashmap
return new DefaultHttpHeaders("create");
}
public String getId()
{
return this.id;
}
public ArrayList<OptionRequestRest> getModel()
{
return this.model;
}
public ArrayList<OptionRequestRest> getOptionRequests()
{
#SuppressWarnings("unchecked")
ArrayList<OptionRequestRest> lReturn = (ArrayList<OptionRequestRest>) this.getSession().get("optionRequest");
return lReturn;
}
// Handles /option-request GET requests
public HttpHeaders index()
{
this.model = this.getOptionRequests();
return new DefaultHttpHeaders("index").lastModified(new Date());
}
public void setId(String pId)
{
this.id = pId;
}
public void setModel(ArrayList<OptionRequestRest> pModel)
{
this.model = pModel;
}
// Handles /option-request/{id} GET requests
public HttpHeaders show()
{
this.model = this.getOptionRequests();
return new DefaultHttpHeaders("show").lastModified(new Date());
}
}
One of the things which is confusing me is that this code works fine and returns the correct object type if the model is not an array. Please let me know if my question is not clear enough and needs additional information. Thanks.