Related
I have the following hosting targets in my firebase.json file
{
"hosting": [
{
"target": "staging",
"public": "build",
"ignore": [
"firebase.json",
"**/.*",
"**/node_modules/**"
],
"rewrites": [
{
"source": "**",
"destination": "/index.html"
}
]
},
{
"target": "production",
"public": "build",
"ignore": [
"firebase.json",
"**/.*",
"**/node_modules/**"
],
"rewrites": [
{
"source": "**",
"destination": "/index.html"
}
]
}
]
}
And my firebaserc file contains the following:
{
"projects": {
"default": "project-name177137"
},
"targets": {
"project-name177137": {
"hosting": {
"staging": [
"project-name177137"
],
"production": [
"productionName"
]
}
}
}
}
If I want to deploy to all I usually just do firebase deploy.
Now imagine I want to deploy to only staging for testing, what firebase command can I use to achieve that?
Thank you.
firebase deploy --only hosting:staging
documentation
I am trying to create EMR-5.30.1 clusters with applications such as Hadoop, livy, Spark, ZooKeeper, and Hive with the help of the CloudFormation template. But the issue is with this template is I am able the cluster with only one application from the above list of applications.
below is the CloudFormation Template
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "Best Practice EMR Cluster for Spark or S3 backed Hbase",
"Parameters": {
"EMRClusterName": {
"Description": "Name of the cluster",
"Type": "String",
"Default": "emrcluster"
},
"KeyName": {
"Description": "Must be an existing Keyname",
"Type": "String",
"Default": "keyfilename"
},
"MasterInstanceType": {
"Description": "Instance type to be used for the master instance.",
"Type": "String",
"Default": "m5.xlarge"
},
"CoreInstanceType": {
"Description": "Instance type to be used for core instances.",
"Type": "String",
"Default": "m5.xlarge"
},
"NumberOfCoreInstances": {
"Description": "Must be a valid number",
"Type": "Number",
"Default": 1
},
"SubnetID": {
"Description": "Must be Valid public subnet ID",
"Default": "subnet-ee15b3e0",
"Type": "String"
},
"LogUri": {
"Description": "Must be a valid S3 URL",
"Default": "s3://aws/elasticmapreduce/",
"Type": "String"
},
"S3DataUri": {
"Description": "Must be a valid S3 bucket URL ",
"Default": "s3://aws/elasticmapreduce/",
"Type": "String"
},
"ReleaseLabel": {
"Description": "Must be a valid EMR release version",
"Default": "emr-5.30.1",
"Type": "String"
},
"Applications": {
"Description": "Please select which application will be installed on the cluster this would be either Ganglia and spark, or Ganglia and s3 backed Hbase",
"Type": "String",
"AllowedValues": [
"Spark",
"Hbase",
"Hive",
"Livy",
"ZooKeeper"
]
}
},
"Mappings": {},
"Conditions": {
"Spark": {
"Fn::Equals": [
{
"Ref": "Applications"
},
"Spark"
]
},
"Hbase": {
"Fn::Equals": [
{
"Ref": "Applications"
},
"Hbase"
]
},
"Hive": {
"Fn::Equals": [
{
"Ref": "Applications"
},
"Hive"
]
},
"Livy": {
"Fn::Equals": [
{
"Ref": "Applications"
},
"Livy"
]
},
"ZooKeeper": {
"Fn::Equals": [
{
"Ref": "Applications"
},
"ZooKeeper"
]
}
},
"Resources": {
"EMRCluster": {
"DependsOn": [
"EMRClusterServiceRole",
"EMRClusterinstanceProfileRole",
"EMRClusterinstanceProfile"
],
"Type": "AWS::EMR::Cluster",
"Properties": {
"Applications": [
{
"Name": "Ganglia"
},
{
"Fn::If": [
"Spark",
{
"Name": "Spark"
},
{
"Ref": "AWS::NoValue"
}
]
},
{
"Fn::If": [
"Hbase",
{
"Name": "Hbase"
},
{
"Ref": "AWS::NoValue"
}
]
},
{
"Fn::If": [
"Hive",
{
"Name": "Hive"
},
{
"Ref": "AWS::NoValue"
}
]
},
{
"Fn::If": [
"Livy",
{
"Name": "Livy"
},
{
"Ref": "AWS::NoValue"
}
]
},
{
"Fn::If": [
"ZooKeeper",
{
"Name": "ZooKeeper"
},
{
"Ref": "AWS::NoValue"
}
]
}
],
"Configurations": [
{
"Classification": "hbase-site",
"ConfigurationProperties": {
"hbase.rootdir":{"Ref":"S3DataUri"}
}
},
{
"Classification": "hbase",
"ConfigurationProperties": {
"hbase.emr.storageMode": "s3"
}
}
],
"Instances": {
"Ec2KeyName": {
"Ref": "KeyName"
},
"Ec2SubnetId": {
"Ref": "SubnetID"
},
"MasterInstanceGroup": {
"InstanceCount": 1,
"InstanceType": {
"Ref": "MasterInstanceType"
},
"Market": "ON_DEMAND",
"Name": "Master"
},
"CoreInstanceGroup": {
"InstanceCount": {
"Ref": "NumberOfCoreInstances"
},
"InstanceType": {
"Ref": "CoreInstanceType"
},
"Market": "ON_DEMAND",
"Name": "Core"
},
"TerminationProtected": false
},
"VisibleToAllUsers": true,
"JobFlowRole": {
"Ref": "EMRClusterinstanceProfile"
},
"ReleaseLabel": {
"Ref": "ReleaseLabel"
},
"LogUri": {
"Ref": "LogUri"
},
"Name": {
"Ref": "EMRClusterName"
},
"AutoScalingRole": "EMR_AutoScaling_DefaultRole",
"ServiceRole": {
"Ref": "EMRClusterServiceRole"
}
}
},
"EMRClusterServiceRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"elasticmapreduce.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"ManagedPolicyArns": [
"arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceRole"
],
"Path": "/"
}
},
"EMRClusterinstanceProfileRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"ec2.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"ManagedPolicyArns": [
"arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceforEC2Role"
],
"Path": "/"
}
},
"EMRClusterinstanceProfile": {
"Type": "AWS::IAM::InstanceProfile",
"Properties": {
"Path": "/",
"Roles": [
{
"Ref": "EMRClusterinstanceProfileRole"
}
]
}
}
},
"Outputs": {}
}
Also, I want to add a bootstrap script in this template as well, Can anyone please help me with the issue.
As per my knoweldge and understanding, Applications in your case should be an array like below, as mentioned in documentation
"Applications" : [ Application, ... ],
In you case, you can list applications like
"Applications" : [
{"Name" : "Spark"},
{"Name" : "Hbase"},
{"Name" : "Hive"},
{"Name" : "Livy"},
{"Name" : "Zookeeper"},
]
For more arguments other than Name to individual application dictionary , see detail here, you can pass Args, Additional_info etc
You can use following way:-
If you set "ReleaseLabel" then there is no need to mention versions of applications
"Applications": [{
"Name": "Hive"
},
{
"Name": "Presto"
},
{
"Name": "Spark"
}
]
For bootstrap:-
"BootstrapActions": [{
"Name": "setup",
"ScriptBootstrapAction": {
"Path": "s3://bucket/key/Bootstrap.sh"
}
}]
Define like this to create all applications at once.
{
"Type": "AWS::EMR::Cluster",
"Properties": {
"Applications": [
{
"Name": "Ganglia"
},
{
"Name": "Spark"
},
{
"Name": "Livy"
},
{
"Name": "ZooKeeper"
},
{
"Name": "JupyterHub"
}
]
}
}
I am trying to flatten a nested JSON returned from a Rest source. The pipeline code is as follows.
The problem here is this pipeline returns only first object from JSON dataset and skips all the rest of the rows.
Can you please guide me on how to iterate over nested objects.
Thanks
Sameet
{
"name": "STG_NCR2",
"properties": {
"activities": [
{
"name": "Copy data1",
"type": "Copy",
"dependsOn": [],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"source": {
"type": "RestSource",
"httpRequestTimeout": "00:01:40",
"requestInterval": "00.00:00:00.010",
"requestMethod": "GET",
"additionalHeaders": {
"OData-MaxVersion": "4.0",
"OData-Version": "4.0",
"Prefer": "odata.include-annotations=*"
}
},
"sink": {
"type": "AzureSqlSink"
},
"enableStaging": false,
"translator": {
"type": "TabularTranslator",
"mappings": [
{
"source": {
"path": "$['value'][0]['tco_ncrid']"
},
"sink": {
"name": "NCRID"
}
},
{
"source": {
"path": "['tco_name']"
},
"sink": {
"name": "EquipmentSerialNumber"
}
}
],
"collectionReference": "$['value'][0]['tco_ncr_tco_equipment']"
}
},
"inputs": [
{
"referenceName": "Rest_PowerApps_NCR",
"type": "DatasetReference"
}
],
"outputs": [
{
"referenceName": "Prestaging_PowerApps_NCREquipments",
"type": "DatasetReference"
}
]
}
],
"annotations": []
}
}
The JSON is in the following format
[
{
"value":[
{
"tco_ncrid":"abc-123",
"tco_ncr_tco_equipment":[
{
"tco_name":"abc"
}
]
},
{
"tco_ncrid":"abc-456",
"tco_ncr_tco_equipment":[
{
"tco_name":"xyz"
},
{
"tco_name":"yzx"
}
}
]
]
}
]
This can be resolved by amending the translator property as follows.
"translator": {
"type": "TabularTranslator",
"mappings": [
{
"source": {
"path": "$.['value'][0].['tco_ncrid']"
},
"sink": {
"name": "NCRID",
"type": "String"
}
},
{
"source": {
"path": "$.['value'][0].['tco_text_id']"
},
"sink": {
"name": "EquipmentDescription",
"type": "String"
}
},
{
"source": {
"path": "['tco_name']"
},
"sink": {
"name": "EquipmentSerialNumber",
"type": "String"
}
}
],
"collectionReference": "$.['value'][*].['tco_ncr_tco_equipment']"
}
This code forces the pipeline to iterate over nested array but as you can see that the NCRID is hardcoded to first element of the value array. This is not exactly what I want as I am looking for all Equipment Serial Numbers against every NCRID. Still researching...
I have been trying desperately for 5 days to create an elasticsearch watcher alert that sends a notification on an incoming webhook teams. However, the answer I receive is "Bad payload received by generic incoming webhook". I do not understand why it does not work.
{
"trigger": {
"schedule": {
"interval": "2m"
}
},
"input": {
"simple": {
"summary": "Test Nom",
"text": "test"
}
},
"condition": {
"always": {}
},
"actions": {
"MS_TEAMS_PORT443": {
"webhook": {
"scheme": "https",
"host": "outlook.office.com",
"port": 443,
"method": "post",
"path": "/webhook/XYZ",
"params": {},
"headers": {
"content-type": "application/json"
},
"body": "{{#toJson}}ctx.payload.summary{{/toJson}}"
}
}
}
}
And this is the response when I launch it:
{
"watch_id": "_inlined_",
"node": "QUApyNq4S5GyhHF-CuNjfg",
"state": "executed",
"status": {
"state": {
"active": true,
"timestamp": "2019-10-21T08:40:39.802Z"
},
"last_checked": "2019-10-21T08:40:39.802Z",
"last_met_condition": "2019-10-21T08:40:39.802Z",
"actions": {
"MS_TEAMS_PORT443": {
"ack": {
"timestamp": "2019-10-21T08:40:39.802Z",
"state": "awaits_successful_execution"
},
"last_execution": {
"timestamp": "2019-10-21T08:40:39.802Z",
"successful": false,
"reason": "received [400] status code"
}
}
},
"execution_state": "executed",
"version": -1
},
"trigger_event": {
"type": "manual",
"triggered_time": "2019-10-21T08:40:39.802Z",
"manual": {
"schedule": {
"scheduled_time": "2019-10-21T08:40:39.802Z"
}
}
},
"input": {
"simple": {
"summary": "Test Nom",
"text": "test"
}
},
"condition": {
"always": {}
},
"metadata": {
"name": "testJsonALaMano",
"xpack": {
"type": "json"
}
},
"result": {
"execution_time": "2019-10-21T08:40:39.802Z",
"execution_duration": 125,
"input": {
"type": "simple",
"status": "success",
"payload": {
"summary": "Test Nom",
"text": "test"
}
},
"condition": {
"type": "always",
"status": "success",
"met": true
},
"actions": [
{
"id": "MS_TEAMS_PORT443",
"type": "webhook",
"status": "failure",
"reason": "received [400] status code",
"webhook": {
"request": {
"host": "outlook.office.com",
"port": 443,
"scheme": "https",
"method": "post",
"path": "/webhook/XYZ",
"headers": {
"content-type": "application/json"
},
"body": "Test Nom"
},
"response": {
"status": 400,
"headers": {
"date": [
"Mon, 21 Oct 2019 08:40:38 GMT"
],
"content-length": [
"49"
],
"expires": [
"-1"
],
"x-beserver": [
"VI1PR07MB5053"
],
"x-aspnet-version": [
"4.0.30319"
],
"x-proxy-backendserverstatus": [
"400"
],
"x-cafeserver": [
"VE1PR03CA0023.EURPRD03.PROD.OUTLOOK.COM"
],
"x-calculatedbetarget": [
"VI1PR07MB5053.eurprd07.prod.outlook.com"
],
"request-id": [
"6d651f70-74b5-4010-a2a6-662666fa9985"
],
"pragma": [
"no-cache"
],
"x-msedge-ref": [
"Ref A: C6E8A3DCFF9541DD95D63FD71ACD695C Ref B: PAR02EDGE0513 Ref C: 2019-10-21T08:40:39Z"
],
"x-feserver": [
"VE1PR03CA0023"
],
"x-powered-by": [
"ASP.NET"
],
"x-backendhttpstatus": [
"400"
],
"content-type": [
"text/plain; charset=utf-8"
],
"cache-control": [
"no-cache"
]
},
"body": "Bad payload received by generic incoming webhook."
}
}
}
]
},
"messages": []
}
Wrong body statement. Need to send text in body. Change your body to be like this
"body": "{\"text\": \"{{ctx.payload.summary}}\"}"
source property is your friend here:
"body": {
"source": {
"text" : "Test Nom"
}
}
I am trying to set up an environment where the first time a user logs into chrome they have a list of pre-determined bookmarks and extensions installed. I have followed Chromiums guide to this as much as possible however seem to be failing when it comes to the granted_permissions section, in an ideal world the user would open Chrome and not be prompted with "This extension requires new permissions". From what i can gather this is through the granted_permissions entry however these do not appear to be parsing through to chrome, ive included a snapshot of the code below:
{
"homepage": "MY_URL",
"homepage_is_newtabpage": false,
"extensions": {
"settings": {
"hdokiejnpimakedhajhdlcegeplioahd": {
"location": 1,
"manifest": {
"key": "MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCiL9y2jziKp5kjb13uYG1bAXsuemUKAYUAwR/N9qTuhMIuly02Ecm63dOkn+M2r26IGfDE/lbPv/dB/W+d34pqGo5uJJY+Omt9t3xsIuz7mQwuvF1H5ozj0OHok5XDFRaBIfPa06RhQw3M7sSZJvQ+qqD3+dr0aLX+mvi0LQ11uQIDAQAB",
"name": "LastPass",
"update_url": "http://clients2.google.com/service/update2/crx",
"granted_permissions": {
"api": [ "contextMenus", "idle", "notifications", "tabs", "unlimitedStorage", "webRequest", "webRequestBlocking" ],
"explicit_host": [ "http://*/*", "https://*/*" ],
"scriptable_host": [ "file:///*", "http://*/*", "https://*/*", "https://1min-ui-prod.service.lastpass.com/*" ]
},
"version": "0.0"
},
"path": "hdokiejnpimakedhajhdlcegeplioahd\\0.0",
"state": 1
},
"cjpalhdlnbpafiamejdnhcphjbkeiagm": {
"location": 1,
"manifest": {
"key": "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAmJNzUNVjS6Q1qe0NRqpmfX/oSJdgauSZNdfeb5RV1Hji21vX0TivpP5gq0fadwmvmVCtUpOaNUopgejiUFm/iKHPs0o3x7hyKk/eX0t2QT3OZGdXkPiYpTEC0f0p86SQaLoA2eHaOG4uCGi7sxLJmAXc6IsxGKVklh7cCoLUgWEMnj8ZNG2Y8UKG3gBdrpES5hk7QyFDMraO79NmSlWRNgoJHX6XRoY66oYThFQad8KL8q3pf3Oe8uBLKywohU0ZrDPViWHIszXoE9HEvPTFAbHZ1umINni4W/YVs+fhqHtzRJcaKJtsTaYy+cholu5mAYeTZqtHf6bcwJ8t9i2afwIDAQAB",
"name": "uBlock Origin",
"update_url": "http://clients2.google.com/service/update2/crx",
"version": "0.0"
},
"path": "cjpalhdlnbpafiamejdnhcphjbkeiagm\\0.0",
"state": 1
}
}
},
"session": {
"restore_on_startup": 1,
"startup_urls": [
"MY_URL/"
]
},
"browser": {
"show_home_button": true,
"check_default_browser": false
},
"bookmark_bar": {
"show_on_all_tabs": true
},
"distribution": {
"show_welcome_page": false,
"skip_first_run_ui": true,
"import_history": false,
"import_bookmarks_from_file": "/Library/Google/bookmarks.html",
"import_bookmarks": false,
"import_home_page": false,
"import_search_engine": false
},
"sync_promo": {
"user_skipped": true
}
}
After opening chrome even with those permissions in place it still disables the extensions pending further permissions however im not really sure what else its after.
Wrong structure. If you look at the docs, it should be as follows:
{
"extensions": {
"settings": {
"mihcahmgecmbnbcchbopgniflfhgnkff": {
"location": 1,
"manifest": {
...
},
"granted_permissions": {
"api": [ "tabs" ],
"explicit_host": [ "http://*.google.com/*", "https://*.google.com/" ],
"scriptable_host": [ "http://example.com/" ]
},
...
},
...
}
}
}
However, you have put the "granted_permissions" key inside, and not on the same level as the "manifest" key.