With the gcloud command line tool I can do:
$ gcloud compute instances list --filter='tags.items:development'
The documentation claims: "..you can also filter on nested fields. For example, you could filter on instances that have set the scheduling.automaticRestart field to true. Use filtering on nested fields to take advantage of labels to organize and search for results based on label values." But no examples are provided, so it's not clear how one actually goes about this.
I've tried labels.development eq *.*, labels eq *development*, labels:development et al.. I've also tried setting the verbosity of the of the command line client to info and looking through the output, as well as monitoring requests that go to the API from the Compute Engine web console, but neither has gotten me anywhere.
Finding Tags with regular expression filters
I'm struggling with the same issue but I think that regular expressions solve the problem.
I have many instances with multiple tags but I can search across all tags with the '~' operator e.g. to find all servers with the production tag:
gcloud compute instances list --filter='tags.items~^production$'
For many servers the 'production' tag is the third entry in tags.items yet the regexp finds it.
This seems to work but I can't find any documentation that specifically says that it should work. The nearest is the section on topic filters which mentions this
key ~ value True if key matches the RE (regular expression) pattern
value.
You can also search for multiple tags
gcloud compute instances list --filter='tags.items~^production$ AND tags.items~^european$'
which would find all servers with the two tags 'production' and 'european'
Tags v Custom metadata
If you want something a bit more flexible than tags (which can only be present or missing), you can attach your own custom multi-valued metadata to an instance (via the UI, command-line or API). You can then search for particular values of that item.
For example suppose I have different instances supporting eCommerce for different brands, I could attach a custom 'brand' metadata item to each server and then find all of the servers which run my "Coca-Cola" brands via ..
gcloud compute instances list --filter="metadata.items.key['brand']['value']='Coca-Cola'"
... and my 'Pepsi Cola' servers with ...
gcloud compute instances list --filter="metadata.items.key['brand']['value']='Pepsi Cola'"
Finding Metadata with regular expression filters
You probably guessed this already but the regular expression operator also works with metadata filters so you can do
gcloud compute instances list --filter="metadata.items.key['brand']['value']~'Cola'"
As explained here you can use the following syntax using gcloud cli:
gcloud compute instances list --filter labels.env=dev
Multi Filtering example that I'm using:
gcloud compute instances list --filter="zone:( europe-west1-d )" --filter="name:( testvm )" --filter labels.group=devops --filter labels.environment_type=production
Add instance metadata MyKey=MyValue for gke-cluster-asia-eas-default-pool-dc8f484c-knbs:
gcloud compute instances add-metadata gke-cluster-asia-eas-default-pool-dc8f484c-knbs --metadata=MyKey=MyValue
Display all instances with MyKey=MyValue:
gcloud compute instances list --filter="metadata.items.key['MyKey'][value]='MyValue'"
Display all instances that belong to the cluster cluster-asia-east1-a:
gcloud compute instances list --filter="metadata.items.key['cluster-name'][value]='cluster-asia-east1-a'"
NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS
gke-cluster-asia-eas-default-pool-dc8f484c-knbs asia-east1-a n1-standard-1 10.140.0.2 104.155.227.25 RUNNING
gke-cluster-asia-eas-default-pool-dc8f484c-x8cv asia-east1-a n1-standard-1 10.140.0.3 104.199.226.16 RUNNING
gke-cluster-asia-eas-default-pool-dc8f484c-z5wv asia-east1-a n1-standard-1 10.140.0.4 104.199.134.9 RUNNING
Related
I want to create system of n agents. All agents are generate random Integer value. My goal is calculating average of these n numbers.
My simple idea of algorithm:
Every Agent sends message with its number to other agents
Every Agent calculates average number
Problems:
I just can't understand how I can create a variable number of agents
How I can take output result
Maybe somebody know how I can do this?
The examples online tend to focus on using the Boot class:
java -cp jade.jar jade.Boot -agents agentName:org.agents.MyAgentClass
You could spawn more agents simply by adding more to the -agents option command-line args (separated by semi-colons):
java -cp jade.jar jade.Boot -agents \
agent1:org.agents.MyAgentClass;agent2:org.agents.MyAgentClass
If you need a variable number of agents, you could move this to a bash script that appends more agents depending on a parameter.
If you really want to go crazy, you can create your own container and add agents to it from your own code and bypass the Boot class. Since your use case is so simple, I don't know that this would be a good way to go yet.
I'm creating instance on the Google Compute Engine with jclouds, using the command:
Template template = context.getComputeService().templateBuilder().imageId("debian-7-wheezy-v20150710").build();
Set<? extends NodeMetadata> listNode = context.getComputeService().createNodesInGroup("teste", 1, template);
But is the error below:
2015-08-16 07:27:04 INFO compute:64 - Image debian-7-wheezy-v20150710 not found in the image cache. Trying to get it from the provider...
Exception in thread "main" java.util.NoSuchElementException: imageId(debian-7-wheezy-v20150710) not found
at org.jclouds.compute.domain.internal.TemplateBuilderImpl.throwNoSuchElementExceptionAfterLoggingImageIds(TemplateBuilderImpl.java:764)
at org.jclouds.compute.domain.internal.TemplateBuilderImpl.findImageWithId(TemplateBuilderImpl.java:745)
at org.jclouds.compute.domain.internal.TemplateBuilderImpl.build(TemplateBuilderImpl.java:688)
at br.com.clouddeploy.main.TestGoogle.main(TestGoogle.java:47)
Any suggestion?
If you are using imageId() you probably need to use the full path to the image. Here's how to find it:
% gcloud compute images list --uri | grep debian-7
shows:
https://www.googleapis.com/compute/v1/projects/debian-cloud/global/images/backports-debian-7-wheezy-v20150710
https://www.googleapis.com/compute/v1/projects/debian-cloud/global/images/debian-7-wheezy-v20150710
So you can use either of these. The one that starts with backports has newer versions of packages, but is still based on Debian 7 Wheezy.
Alternatively, it looks like jclouds has the ability to do this URL creation on its own, but instead of using imageId(), you should use (see GitHub code example):
ComputeService compute = initComputeService(account, credentials);
[...]
TemplateBuilder templateBuilder = compute.templateBuilder();
templateBuilder.fromImage(compute
.getImage("debian-7-wheezy-v20140408"));
The root problem is that I have a project that has many tens of thousands of operations, such that it's very difficult to track down operations associated with specific instances being created or deleted. So, I'm looking for a way to list operations by target.
The deprecated gcutil docs talk about setting filter expressions to possibly filter operations by target instance name or instance selfLink, but there doesn't seem to be any mention of a similar filter expression in gcloud compute operations list described here:
gcloud compute operations list [NAME …] [--format FORMAT] [--help] [--limit LIMIT] [--log-http] [--project PROJECT_ID] [--quiet, -q] [--regexp REGEXP, -r REGEXP] [--sort-by SORT_BY] [--trace-token TRACE_TOKEN] [--uri] [--global | --regions [REGION,…] | --zones [ZONE,…]] [-h]
The only non-global flags listed there are the following:
--global
If provided, only global resources are shown.
--limit LIMIT
The maximum number of results.
--regexp REGEXP, -r REGEXP
A regular expression to filter the names of the results on. Any names that do not match the entire regular expression will be filtered out.
--regions [REGION,…]
If provided, only regional resources are shown. If arguments are provided, only resources from the given regions are shown.
--sort-by SORT_BY
A field to sort by. To perform a descending-order sort, prefix the value of this flag with a tilde (~).
--uri
If provided, the list command will only print URIs for the resources returned. If this flag is not provided, the list command will print a human-readable table of useful resource data.
--zones [ZONE,…]
If provided, only zonal resources are shown. If arguments are provided, only resources from the given zones are shown.
Furthermore, the examples given by the gcutil docs don't seem to work:
$ gcutil listoperations --zone us-east1-a --filter="target eq '.*dhuo.*'"
WARNING: 'gcutil' has been deprecated and soon will be removed from Cloud SDK distribution.
WARNING: Please use 'gcloud compute' instead.
WARNING: For more information see https://cloud.google.com/compute/docs/gcutil
Error: Invalid value for field 'filter': 'target eq '.*dhuo.*''. Unrecognized field name in list filter: target.
That's despite the raw API docs describing 'target' as an example here:
Filter expression for filtering listed resources, in the form filter={expression}. Your {expression} must contain the following:
FIELD_NAME COMPARISON_STRING LITERAL_STRING
FIELD_NAME: The name of the field you want to compare. The field name must be valid for the type of resource being filtered. Only atomic field types are supported (string, number, boolean). Array and object fields are not currently supported. For example, if you search for machineType in the Firewalls collection, the filter will fail, because the machineType field is specific to instances and other resources do not have that property.
COMPARISON_STRING: The comparison string, either eq (equals) or ne (not equals).
LITERAL_STRING: The literal string value to filter to. The literal value must be valid for the type of field (string, number, boolean). For string fields, the literal value is interpreted as a regular expression using RE2 syntax. The literal value must match the entire field. For example, when filtering instances by name, name eq my-instance won't work, but name eq .*my-instance will work.
For example, for Operation resources, you could filter by the type of the operation:
filter=operationType ne insert
The above filter returns only results whose type field does not equal insert. You can also enclose your literal string in single, double, or no quotes. For example, all three of the following would be valid expressions:
filter=operationType ne "insert"
filter=operationType ne 'insert'
filter=operationType ne insert
If you use a complex regular expression, you need to encode the special characters, including quotes. Consider the following regular expression:
target eq 'example-name-[0-9]+'
To use the expression, you would need to encode it:
filter=target+eq+%27example-name-%5B0-9%5D%2B%27
If you are using the API through a client, such as client libraries, or gcloud compute, keep in mind that your client might encode your regular expression automatically so that you will not need to encode it yourself.
Is there something I'm doing wrong, and is it indeed possible to filter zone operations based on the name of the instance I'm trying to insert?
I'm not sure how to do it using gcloud, but you can still use gcutil:
gcutil listoperations --zone us-east1-a --filter="targetLink eq '.*dhuo.*'"
So it is almost the same as your, but should use 'targetLink' instead of 'target'.
You could also do something like this if you wanted to use gcloud and search specifically for the target:
$ gcloud compute operations list | awk '$3 ~ /dhuo\./ {print $0}'
I have a list of DNs, and for performance reasons I want to retrieve the attributes of every DN in the list in a single trip to the LDAP server.
Seems like searching by DN, i.e., using DN as a filter search, is not possible
Using DN in Search Filter
http://www.openldap.org/lists/openldap-software/200503/msg00520.html
....is there any alternative?
Sure you can.
ldapsearch -h <ldaphost> -b "cn=joe,dc=yourdoamin,dc=com" -s base -D cn=admin,dc=yourdomain,dc=com -W "(objectclass=*)" "*"
Will retrieve all user attributes for the DN: cn=joe,dc=yourdoamin,dc=com.
But, for the list, you would need to repeat the search for each one.
We often do this in a bash script.
Can you use a filter to identify which DNs you need?
-jim
Seems like it is only possible in Active Directory. All I had to do is filter by the distinguishedName attribute, however on my tests there was no performance gain.
Active Directory includes the distinguishedName attribute on every
object; the value is the object's DN. The following example elaborates
the previous example to include a value of distinguishedName on each
object.
http://msdn.microsoft.com/en-us/library/cc223167.aspx
How to request the number of nodes (not procs), while job submission in SGE?
for e.g. In TORQUE, we can specify qsub -l nodes=3
How to request the nodes by their names in SGE?
for e.g. In TORQUE, we can do this by qsub -l nodes=abc+xyz+pqr, where abc, xyz and pqr are hostnames
For single hostname, qsub -l hostname=abc it works. But how do I delimit multiple hostnames in SGE?
Requesting the number of nodes with Grid Engine is done indirectly.
When you want to submit a parallel job then you have to request
a parallel environment (man sge_pe) together with the amount
of slots (processors etc) like qsub -pe mytestpe 12...
Depending on the allocation_rule defined in the parallel environment
(qconf -sp mytestpe) the slots are distributed over one or more
nodes. If you have a so called fixed allocation rule where you
just add a certain number as allocation rule like 4 (4 slots per
host) it is easy. If you like one host just submit with -pe mytestpe 4
if you want 10 nodes just submit with -pe mytestpe 40.
Node name can be requested by the -l h=abc. Since node names are
RESTRINGS (regular expression strings) in Grid Engine you can create
a regular expression for host filtering: qsub -l h="abc|xyz".
You can also create host groups (qconf -ahgrp) and request
so called queue domains (qsub -q all.q##mygroup).
Daniel
http://www.gridengine.eu
you can use -tc to limit the number of concurrent tasks (i.e., number of slots that will be used for an array job). I use this when I submit array jobs with 100 sub-jobs to limit the impact on our queue, defaulting to 10 simultaneous jobs with -tc 10. As each job finishes, another array job from the pending pool will be submitted.
the only way I've been able to figure out to do this would be to set up specific resource quota sets (using qconf -mrqs) specifying the particular host groups you want to use. You would have to set up all of the combinations that you want, first. I don't see a real reason to specify specific hosts, though, unless these hosts have specific resources that you want to use (in which case, I'd set up consumable resources for those and apply the appropriate number of resources to each host that can supply them, then use that instead of specifying the specific hosts for a particular job).