how can I match these regular expressions? - mysql

I have a url stored in a database. From a given url, i have to fetch data from the database but I'm confused how can I indentify the actual url. It's confusing when the url from the user is dynamic like text or a number.
example:
/user should match the url /user
/user/1 should match the url /user/{id}
/user/name/johndoe should match the url /user/name/{name}
/user/1/johndoe should match the url /user/{id}/{name}
I have tried this one it works for /user/1 i.e with integers but I cannot make it work with string. There's no way to indentify the string parameters. Or is there other way in mysql a workaround.
MOTO: Fetch the content from the database where a given url string matches the url column. Note: the {id}, {name}, {type} are dynamic values.
Controller:
$sanitizedUrl = preg_replace('/\/*\d{1,}/','/{id}', '/user/name/1');
UrlContent::where('url', $sanitizedUrl)->first()->content;
This code can only work with the urls which have numbers only. Example: /user/12/type/12 is replaced as /user/{id}/type/{id}
Database : url_contents
| url | content |
-----------------------------------------------------------
| /user/name/{id} | matches if url is /user/name/1 |
| /user/{id}/{name} | if url is /user/1/john |
| /user/{type}/{name} | if url i /user/{type}/{name} |

You should do it the other way:
function getUrlContent($url)
{
$replacements = [
'{id}' => '\d+',
'{name}' => '\w+',
];
$urlContents = UrlContent::all();
foreach ($urlContents as $urlContent) {
if (preg_match('~' . strtr($urlContent->url, $replacements) . '~', $url)) {
return $urlContent->content;
}
}
return null;
}
$url = '/user/1/johndoe';
var_dump(getUrlContent($url));
Also you may need to cache all built regular expressions for a time to reduce mysql overhead of a roundtrip. If url_contents table grows significantly you may have to add another column which will contain the regular expression itself:
+-----------------+----------------+------------+
| url | url_regex | content |
+-----------------+----------------+------------+
| /user/name/{id} | /user/name/\d+ | ......... |
+-----------------+----------------+------------+

Related

Sentinel KQL JSON with Dynamic Label

I'm experimenting with Microsoft Sentinel and trying to understand how to parse JSON elements. One experiment is that I've wired my house with temperature and humidity sensors and fed them in, now the difficulty is the parsing... they're syslog events with a Message containing JSON as shown below.
SENSOR =
{
"ZbReceived":
{
"0x03FA":
{
"Device":"0x03FA",
"Name":"2_Back_Bedroom",
"Humidity":71.66,"Endpoint":1,
"LinkQuality":66
}
}
}
Unfortunately the devices include the device ID as a label in the JSON, which makes it hard for me to figure out how to extract all the fields. There are 8 sensors, so repeating this for every one of them seems inefficient, but maybe it's necessary?
Is there a way I could extract the values from 8 different sensors? I've tried .[0]. and other variants, but no luck.
print T = dynamic('SENSOR = {"ZbReceived":{"0x03FA":{"Device":"0x03FA","Name":"2_Back_Bedroom","Humidity":71.66,"Endpoint":1,"LinkQuality":66}}}')
| mv-expand humidity = parse_json(substring(T, 9)).ZbReceived.["0x03FA"].Humidity
| mv-expand device = parse_json(substring(T, 9)).ZbReceived.["0x03FA"].Device
| mv-expand name = parse_json(substring(T, 9)).ZbReceived.["0x03FA"].Name
| mv-expand battery = parse_json(substring(T, 9)).ZbReceived.["0x03FA"].Battery
| mv-expand temperature = parse_json(substring(T, 9)).ZbReceived.["0x03FA"].Temperature
Quick explanation:
Under
print T = dynamic('SENSOR = {"ZbReceived":{"0x03FA":{"Device":"0x03FA","Name":"2_Back_Bedroom","Humidity":71.66,"Endpoint":1,"LinkQuality":66}}}')
| parse tostring(T) with "SENSOR = " sensor:dynamic
| project device = sensor.ZbReceived[tostring(bag_keys(sensor.ZbReceived)[0])]
| evaluate bag_unpack(device)
Device
Endpoint
Humidity
LinkQuality
Name
0x03FA
1
71.66
66
2_Back_Bedroom
Fiddle
P.S.
For clarity, the line with the project operator could be replaced with the following 2 lines:
| extend device_id = tostring(bag_keys(sensor.ZbReceived)[0]) // e.g., 0x03FA
| project device = sensor.ZbReceived[device_id]

Scenario Outline: Using Whitelists & Ranges?

I have been building up a Cucumber automation framework and have a number of components to test. I have used Scenario Outline's to capture various values and their expected responses.
What I see as a problem:
I have to specify every single type of input of data and the error message to go with it. From the example Scenario Outline below you can see I have certain numbers that all are expected to return the one message. If anything does not equal these values the return an error message:
Scenario Outline: Number is or is not valid
Given I send an event with the "Number" set to <num>
Then I will receive the following <message>
Examples:
| num | message |
| 0 | "Processed event" |
| 1 | "Processed event" |
| 2 | "Processed event" |
| 3 | "Processed event" |
| 4 | "Processed event" |
| 5 | "Processed event" |
| 6 | "Message failed" |
| -1 | "Message failed" |
| "One" | "Message failed" |
What I would like to do:
I would basically like to have a "whitelist" of good data defined in the Scenario Outline and if there is any other value input - it returns the the expected error message. Like the following:
Scenario Outline: Number is or is not valid
Given I send an event with the "Number" set to <num>
Then I will receive the following <message>
Examples:
| num | message |
| 0-5 | "Processed event" |
| Anything else | "Message failed" |
Is the following possible with the code behind it? As you can see it would have benefits of making an automation suite far more concise and maintainable. If so please let me know, keen to discuss.
Thanks!
Kirsty
Cucumber is a tool to support BDD. This means that it works really well when you have to communicate about behavior. But this particular problem is going towards validating the properties of the event validator, i.e. property based testing. So it might be worth to split the test strategy accordingly.
It appears there is a rule that valid events results are processed and invalid events are rejected. This is something you could test with Cucumber. For example:
Feature: Events
This system accepts events. Events are json messages.
Examples of well known valid and invalid json messages
can be found in the supplementary documentation.
Scenario: The system accepts valid events
When a well known valid event is send
Then the system accepts the event
And responds with "Processed event"
Scenario: The system rejects invalid events
When a well known invalid event is send
Then the system rejects the event
And responds with "Message failed"
It also appears there is a rule that valid events have a field "Number" set to any value between 0-5. And since sounds like a json object I'm guessing the strings "0", "1", "2", "3", "4", "5" are also valid. Anything else is invalid.
A good way to test this exhaustively is by using property based testing framework. For example JQwik. Given a description of a set of either valid or invalid values it will randomly try a few. For a simplified example:
package my.example.project;
import net.jqwik.api.*;
import static org.assertj.core.api.Assertions.assertThat;
class ValidatorProperties {
#Provide
Arbitrary<Object> validValues() {
Arbitrary<Integer> validNumbers = Arbitraries.integers().between(0, 5);
Arbitrary<String> validStrings = validNumbers.map(Object::toString);
return Arbitraries.oneOf(validNumbers, validStrings);
}
#Provide
Arbitrary<Object> invalidValues() {
Arbitrary<Integer> invalidNumbers = Arbitraries.oneOf(
Arbitraries.integers().lessOrEqual(-1),
Arbitraries.integers().greaterOrEqual(6)
);
Arbitrary<String> invalidStrings = invalidNumbers.map(Object::toString);
return Arbitraries.oneOf(
invalidNumbers,
invalidStrings,
Arbitraries.just(null)
);
}
#Property
void accepts0To5(#ForAll("validValues") Object value) {
Validator validator = new Validator();
assertThat(validator.isValid(value)).isTrue();
}
#Property
void rejectsAnythingElse(#ForAll("invalidValues") Object value) {
Validator validator = new Validator();
assertThat(validator.isValid(value)).isFalse();
}
static class Validator {
boolean isValid(Object event) {
return event != null && event.toString().matches("^[0-5]$");
}
}
}
Split this way the Cucumber tests describe how the system should respond to valid and invalid events. While the JQwik test describe what the properties of a valid and invalid event are. This allows much more clarity on the first and a greater fidelity on the second.

How do you write an array of numbers to a csv file?

let mut file = Writer::from_path(output_path)?;
file.write_record([5.34534536546, 34556.456456467567567, 345.56465456])?;
Produces the following error:
error[E0277]: the trait bound `{float}: AsRef<[u8]>` is not satisfied
--> src/main.rs:313:27
|
313 | file.write_record([5.34534536546, 34556.456456467567567, 345.56465456])?;
| ------------ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `AsRef<[u8]>` is not implemented for `{float}`
| |
| required by a bound introduced by this call
|
= help: the following implementations were found:
<&T as AsRef<U>>
<&mut T as AsRef<U>>
<Arc<T> as AsRef<T>>
<Box<T, A> as AsRef<T>>
and 44 others
note: required by a bound in `Writer::<W>::write_record`
--> /home/mlueder/.cargo/registry/src/github.com-1ecc6299db9ec823/csv-1.1.6/src/writer.rs:896:12
|
896 | T: AsRef<[u8]>,
| ^^^^^^^^^^^ required by this bound in `Writer::<W>::write_record`
Is there any way to use the csv crate with numbers instead of structs or characters?
Only strings or raw bytes can be written to a file; if you try to give it something else, it isn't sure how to handle the data (as #SilvioMayolo mentioned). You can map your float array to one with strings, and then you will be able to write the string array to the file.
let float_arr = [5.34534536546, 34556.456456467567567, 345.56465456];
let string_arr = float_arr.map(|e| e.to_string();
This can obviously be combined to one line without using the extra variable, but it is a little easier to see the extra step that we need to take if it is split apart.

Kusto KQL reference first object in an JSON array

I need to grab the value of the first entry in a json array with Kusto KQL in Microsoft Defender ATP.
The data format looks like this (anonymized), and I want the value of "UserName":
[{"UserName":"xyz","DomainName":"xyz","Sid":"xyz"}]
How do I split or in any other way get the "UserName" value?
In WDATP/MSTAP, for the "LoggedOnUsers" type of arrays, you want "mv-expand" (multi-value expand) in conjunction with "parsejson".
"parsejson" will turn the string into JSON, and mv-expand will expand it into LoggedOnUsers.Username, LoggedOnUsers.DomainName, and LoggedOnUsers.Sid:
DeviceInfo
| mv-expand parsejson(LoggedOnUsers)
| project DeviceName, LoggedOnUsers.UserName, LoggedOnUsers.DomainName
Keep in mind that if the packed field has multiple entries (like DeviceNetworkInfo's IPAddresses field often does), the entire row will be expanded once per entry - so a row for a machine with 3 entries in "IPAddresses" will be duplicated 3 times, with each different expansion of IpAddresses:
DeviceNetworkInfo
| where Timestamp > ago(1h)
| mv-expand parsejson(IPAddresses)
| project DeviceName, IPAddresses.IPAddress
to access the first entry's UserName property you can do the following:
print d = dynamic([{"UserName":"xyz","DomainName":"xyz","Sid":"xyz"}])
| extend result = d[0].UserName
to get the UserName for all entries, you can use mv-expand/mv-apply:
print d = dynamic([{"UserName":"xyz","DomainName":"xyz","Sid":"xyz"}])
| mv-apply d on (
project d.UserName
)
thanks for the reply, but the proposed solution didn't work for me. However instead I found the following solution:
project substring(split(split(LoggedOnUsers,',',0),'"',4),2,9)
The output of this is: UserName

Custom Function to Extract Data from JSON API Based on Column Values in Excel VBA

I've an excel workbook that looks something like this:
/-------------------------------------\
| Lat | Long | Area |
|-------------------------------------|
| 5.3 | 103.8 | AREA_NAME |
\-------------------------------------/
I also have a JSON api with a url of the following structure:
https://example.com/api?token=TOKEN&lat=X.X&lng=X.X
that returns a JSON object with the following structure:
{ "Area": "AREA_NAME", "OTHERS": "Other_details"}
I tried to implement a VBA function that will help me to extract AREA_NAME. However, I keep getting syntax errors. I don't know where I am going wrong.
Function get_p()
Source = Json.Document (Web.Contents("https://example.com/api?token=TOKEN&lat=5.3&lng=103.8"))
name = Source[Area]
get_p = Name
End Function
I intentionally hardcoded the lat and long value for development purposes. Eventually, I want the function to accept lat and long as parameters. I got the first line of the function from PowerQuery Editor.
Where am I going wrong? How to do this properly in VBA? Or is there a simpler way using PowerQuery?