I am using GE2011.11. Previously, I was using Torque/Maui. In Maui, there is a concept of partition with the help of which we can logically divide the cluster. A node can't belong to more than one partition.
In GE, I am thinking of host groups to achieve this but a node(host) can belong to more than one host group this is what is undesirable. Any idea, how can we achieve partitioning with host groups?
In GE, there is a feature called multi-clustering. Can we use this feature to implement partitioning? Any idea, how do we use this?
With SGE you can modify the host groups manually so that a host(node) only exists in a single host group. You can then create queues that only use a single host group. The users would then submit jobs to the specific queues(which themselves can be limited by users) to access specific host groups. This would effectively divide your grid.
The basic things to look out for when performing this setup relates to the all.q queue and the #allhosts hosts group. If you remove all hosts from the #allhosts group you should be okay to do this.
Let me know if you have further questions, I would be happy to help as I have been administering an SGE cluster for years now.
Since the administrator defines the membership of hostgroups just define the groups so as not to overlap. If you want to control which users can run on which partition you can do this by defining grid engine acls/usersets for each group of users and using the enhanced queue configuration syntax to associate a different list with each hostgroup in the queue definitions.
user_lists chemists,[#physicshosts=physicists],[#biologyhosts=biologists]
If what you are trying to do is ensure a given job runs on one set of hosts or the other but not a mixture then you need to define multiple identical parallel environments and use a similar trick to associate each PE with a different hostgroup.
pe_list mpich-A,[#switchb=mpich-B],[#switchc=mpich-C]
Users then request a wildcard PE when submitting their job:
#$ -pe mpich-* 8
Related
Maybe the question is a bit complex and complicated (that's why I need to make a brief introduction)
my team and I are developing a Mobile App with Node js. Now we are in the part of the database structure. Our idea is to do it in Azure SQL. But we have a couple of questions regarding the structure of the database.
We offer 5 services (at the moment), of which each user can be assigned several services (may non or all). Based on the services it has, the user will be redirected to a screen where all the services will be and only those assigned with color (to be able to click) and the others in gray (so that they cant click it)
Which is better, create one column per service or all services in a single column array style?
for example
service 1| service 2| service 3|service 4|service 5|
true | fasle | true | true| false| true
or
service
[service 1,service 2,service 3,service 4,service 5]
Because I think that if in the future we have x services, going through the entire array and making a condition to verify what service it has is going to make the latency of the app to slow, instead hitting a certain column maybe makes it faster
I hope the question has been understood, sorry if the maries.
regards
This isn't really an azure SQL question but more of a relational database question.
In general you should avoid both these methods and try to normalize your database.
Your database knows how to query multiple databases without any performance hits, its made for it.
The best option in my experience is to create a many to many table connection
So one table that holds the original data without any mention of a service, Maybe called Entities
Id, Data, Time, Active
Another table that holds the relations to the services, called EntitiesToServices
Id, ServiceId, OtherTableId
And a third table that holds data about the services called Services
Id, Name
In this way you can expand all your services freely and add more tables without anyone interfering with each other.
If all you need is a set of up to 64 true/false values, consider a single column of type SET. Similarly you could some any sized INT (again with a limit of 64 flags) and turn on/off each 'service'. Today you have 5; tomorrow, as you say, there will be more.
The syntax for SET is a bit clumsy. So is using INT for this.
It is very compact; this may or may not be a bonus.
Normalizing (as mentioned in another answer) may be a better solution, especially if you need to store more than just on/off for each service for each user.
Please provide more details on what actions will happen with these flags; then we can get into more detail.
I have read many posts and articles that state quite clearly that for the best in worst-case damage limitation, MySQL accounts should only be able to do what they're intended to do, and nothing else. This makes perfect sense and so there would be a connection account to my Database that does all the SELECTing of data, and another account that does UPDATE and INSERT activities, so that if someone does get the compromise by SELECTing data they shouldn't, then they can't quite as easily then UPDATE that data or INSERT etc.
You get the idea.
But, I have various Databases and use accounts to read/SELECT data and the this is output to the client page, but often these accounts will need to be updated, small things such as updating when a (website) user logs into their account, or updating some sort of hitcounter or other minor feature. Given the ringfencing of concerns outlined above, I feel it's a bit like using a flood to put out a campfire, to allow UPDATEing (etc.) to a privileged MySQL connection simply to say that user Bob logged in last at 4:10pm.
I have been digging on the web for suitable guides, blog posts and articles about how to best structure using multiple MySQL privileged accounts to complete the nessecary work with as minimum a risk of excess privilege as possible, but I have found nothing that has been much use, (mostly because of my wording seems to be attracting articles about setting up website users, or other topics associated with these keywords :-/ )
I have a few ideas on current approach and wanted a bit of feedback on the best method for doing activities as described in paragraph 2, typically 95% SELECTing, and a few specific instances of UPDATEing, or if any of the following are possible (or on flipside, are very bad ideas)? :
I currently have seperate PHP connection objects for each connection privilege user. Is this the best approach?
Could I somehow giving a privilege user access to only update a certain table (or even a certain table column?), as well as SELECT from any table? This would be perfect.
Are using TRIGGERs a common approach and would this have any down sides if I created a Trigger (with a privileged user) and then let a SELECT user account access triggers?
Could I set certain users can only use certain triggers?
Is there another way of doing this?
MySQL allows for users to have different privileges set both at database and individual table levels. The documentation on the GRANT (http://dev.mysql.com/doc/refman/5.7/en/grant.html) syntax gives an example of setting all privileges to a user on one database while only select access to a table in another database.
Privileges can even be set for specific columns (http://dev.mysql.com/doc/refman/5.7/en/grant.html#grant-column-privileges) in a table & also for stored procedures (http://dev.mysql.com/doc/refman/5.7/en/grant.html#grant-routine-privileges).
I have different Active Directory groups and one BO universe.
Different active directory groups should have different restricted access on data in the universe.
How can I implement that? (unfortunately I did not found corresponding tutorial or documentation on the net.)
If there are more than one ways to implement data access, what is the best practice?
Thanks.
There are two main ways to implement row-level security in a universe. One is via a Security Profile; the other is via #variable('BOUSER').
If the security is to be applied at a group level (that is, all members of a group should have the same condition applied), then a Security Profile is appropriate. This is covered in Chapter 17 of the IDT User Guide. At a high level, the steps are:
From IDT, launch the Security Editor
Select the universe, and create either a Data Security Profile or a Business Security Profile (read the docs for the differences between the two, but one is that with the DSP, you will write a WHERE condition; with the BSP, you select objects and define a condition)
Once the Security Profile is created, select the group or groups that it should apply to
Repeat the above for each group that should have a restriction applied
The other method for applying row-level security is only applicable when the data source includes a table that has a mapping of BO user IDs to the values that they may have access to. For example, let's say you have a security table in your data source that looks like this:
user_id region
------- ------
U123 NE
U123 SE
U321 W
and your fact table looks like:
pk region value
__ ______ _____
1 NE 3
2 W 4
You can apply security such that user U123 only sees the "NE" row, and user U321 only sees the "W" row. You would join the two tables on region (security.region=fact.region), then create a new mandatory filter on security.user_id=#variable('BOUSER'). This will force the filter to be applied to all queries.
Note that both of the above methods work by adding conditions to the query's WHERE condition. If your users have permission to view and edit the query's SQL, they will be above to override the filtering logic. To ensure security, users should be denied this right.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 9 years ago.
Improve this question
My app involves implementing data and displaying it to the user.i'll give you a full explanation of the app and what i'm trying to achieve.
Basically the app finds out what car is best suited for the user. last week i decided that a drop down menu will be easier and more interactive than a normal questionnaire. i have three labels, performance, luxury, eco friendly and three buttons, high performance, high luxury and high eco friendly. how can i add data to these so that when the user picks these preferences, it links to a specific car, let me remind you that this is just one of many controllers and questions.
I read that MYSQL is good but I'm not sure if its easier or harder. To be honest, im still in beginner level for programming. Also some linking and database videos on youtube are a joke.
I'll try to get you more on track. Your question shows that you're a newbie but if you're a newbie with some sense you'll catch on instead of complain.
MySQL is a variant of SQL. Unfortunately Oracle (evil Microsoft of the business world--think vendor lock-in) bought out Sun Microsystems (a not evil corporation that should have been defended to the death or close-to). There is some political concern that MySQL is slowly being closed off in favor of Oracle so if you have the option opt for PostgreSQL.
With a database you have two things: a database and a user.
The database part is easy, that is where information is stored.
The user part is still pretty easy, different users have different permissions. Those permissions in example are what SQL commands can be executed.
A database's structure is basically....
Database --> Table --> Columns/Rows
A database table is like an HTML table, columns are sets of data (a 'name' column, an 'address' column, etc). Each row is an entry. Bob's information may be on row 34, Sally's information on row 35, etc.
The four most common commands you'll use: SELECT, INSERT, DELETE and UPDATE. With my software unless a webmaster is signed in a lower level privileged SQL user only has access to these four base SQL commands.
There are two contexts to working with SQL: building it and using it. I'm not an expert though what I've generally accepted is that using SQL across different variants (e.g. MySQL and PostgreSQL) for day-to-day use (accessing, updating, removing, editing data) utilizes standardized SQL commands. Sure, some companies may change the structure of their database on a daily or even hourly basis ON a regular basis but generally speaking not everyone. Commands to change a database structure such as ALTER may NOT be cross-variant standard. Why does this matter? If you don't understand the politics of what is involved then YOU DO NOT UNDERSTAND WHAT IS INVOLVED. So I've made sure I'll be able to easily migrate from MySQL to PostgreSQL by avoiding vendor specific syntax. Spend an extra 20% doing things right the first time and you'll save 80% of the time wasted on maintenance by everyone else.
If you want jump in and start using MySQL get a copy of XAMPP. It takes minimal effort and is great for LOCAL testing (local means YOUR computer only accessible to you/your LAN, live means available on the web in general). I'm not sure about xcode but the general instructions should at least give you a basic guide to stay on track in regards to databases.
If you're looking to learn with a GUI you'll want to use phpMyAdmin (included with XAMPP) though I don't recommend downloading the latest copy as they're going way overboard with some things in phpMyAdmin. PostgreSQL has it's equivalent. It'll show you working (albeit VERY messy) examples of commands. For your reference here are some commands I use in the console...
The highest privileged user in SQL is usually called 'root' (without quotes). You'll have root access on your localhost (local) and dedicated servers (likely VPS too) though not on shared servers unless you're some kind of 1337 haxorz.
The four most common commands...
Using SELECT to retrieve data...
SELECT * FROM database_table;
SELECT id, name FROM database_table WHERE cool_factor='exact matching text';
Using DELETE to remove a record...
DELETE FROM database_table WHERE id='1';
Using INSERT to add a single new record...
INSERT INTO email_filters (age, name) VALUES ('Bob');
Using INSERT to add multiple new records...
INSERT INTO email_filters (name) VALUES ('Bob','23'), ('Sally','24'), ('Susan','25'), ('Irate Yeti','9001');
Using UPDATE to update an existing record...
UPDATE accounts SET name='Mr. Yeti', profession='eating people' WHERE id='234';
BTW it's common practice to make your syntax uppercase to differentiate between SOL-specific syntax and your content.
Important commands that help you create and get around in SQL...m
Login: this will prompt you for a password...
mysql -uroot -p
Show a list of databases (phpMyAdmin will let you create databases)...
show databases;
Select a database to work with...
USE mysql;
Select an SQL user...
SELECT User FROM user;
SELECT User FROM mysql.user;
Create a user (with a normal and admin example)...
CREATE USER 'example_v10'#'localhost' IDENTIFIED BY 'my_!1337_p#ss';
CREATE USER 'example_v10a'#'localhost' IDENTIFIED BY 'my_!1337_p#ss';
Before an SQL user can access a database it must be granted permission, here is a limited permission example...
GRANT SELECT, INSERT, UPDATE, DELETE ON `example_mysite_database`.* TO 'example_v10'#'localhost';
...and an administrative user being granted all permissions (use with caution)...
GRANT ALL PRIVILEGES ON `example_mysite_database`.* TO 'example_v10a'#'localhost';
Import an SQL file in to an existing database...
SOURCE C:\path1\path2\the.sql
Describe a database...
DESCRIBE table_name;
Change an SQL user's password...
SET PASSWORD FOR 'root'#'localhost' = PASSWORD('lulz_bad_password');
It's VERY important to realize in the long run that you'll be COMPOSING SQL queries, you NEVER dump them in a loop. Your goal is to make larger more elegant SQL queries that do more in fewer queries if not a single query whenever possible. In example a typical page request on my site's software utilizes about eight~12 queries subjective to the section/page/permissions/etc. Poorly written software like WordPress will utilize dozens if not more requests.
You will absolutely not be able to do anything effectively with databases without learning how to do RELATIONAL SQL; that is just fancy way of saying you know how to use INNER JOIN and LEFT JOIN (and some people use OUTER JOIN). A JOIN allows you to store a piece of information in the entire database once, like an account name. If you build a forum and want to pull forum posts you would JOIN the user accounts table and simply pull the user names by dynamically referencing them, you wouldn't actually store their user name every single time. However that is for another day. I learned SQL over a couple months and once I learned relational SQL I exploded making a blog in three weeks, a forum in a month, a chat room in three weeks, etc.
Feel free to use this as a cheat-sheet and give you some sense in how to reference some technical jargon. when you ask questions you need to be as concise and accurate with terminology as possible. Asking extremely large questions (how can I make a facebook without doing any work?) is bad, asking a specific question is good (How do I have my programming language connect to a database with a specific user that has limited privileges?)
I've been thinking about this all evening (GMT) but I can't seem to figure out a good solution for this one. Here's the case...
I have to create a signup system which distinguishes 4 kinds of "users":
Individual sign ups (require address info)
Group sign ups (don't require address info)
Group contact (require address info)
Application users (don't require address info)
I really cannot come up with a decent way of modeling this into something that makes sense. I'd greatly appreciate it if you could share your ideas.
Thanks in advance!
Sounds like good case for single table inheritance
Requiring certain data is more a function of your application logic than your database. You can definitely define database columns that don't allow NULL values, but they can be set to "" (empty string) without any errors.
As far as how to structure your database, have two separate tables:
User
UserAddress
When you have a new signup that requires contact info, your application will create records in both tables. When a new signup doesn't require address info, your application will only create a record in the User table.
There are a couple considerations here: first, I like to look at User/Group as a case of a Composite pattern. It clearly meets the requirement: you often have to treat the aggregate and individual versions of the entity interchangeably (as you note). Implementing a composite in a database is not that hard. If you are using an ORM, it is pretty simple (inheritance).
On the other part of the question, you always have the ability to create data structures that are mostly empty. Generally, that's a bad idea. So you can say 'well, in the beginning, we don't have any information about the User so we will just leave all the other fields blank.' A better approach is to try and model the phases as if they were part of an FSM. One of the clearest ways to do this in this particular case is to distinguish between Users, Accounts and some other more domain-specific entity, e.g. Subscriber or Customer. Then, I can come and browse using User, sign up and make an Account, then later when you want address and other personal information, become a subscriber. This would also imply inheritance, and you have the added benefit of being able to have a true representation of the population at any time that doesn't require stupid shenanigans like 'SELECT COUNT(*) WHERE _ not null,' etc.
Here's a suggestion from my end after weighing pro's and con's on this model. As I think the ideal setup is to have all users be a user entity that belong to a group without differentiating groups from individuals (except of course flag a group contact person and creating a link with a groups table) we came up with the alternative to copy the group contact user details to the group members when they group is created.
This way all entities that actually are a person will get their own table.
Could this be a good idea? Awaiting your comments :)
I've decided to go with a construction where group members are separated from the user pool anyway. The group members eventually have no relation with a user since they don't require access to mutating their personal data, that's what a group contact person is for. Eventually I could add a possibility for groups to have multiple contact persons, even distinguishing persons that are or are not allowed to edit any member data.
That's my answer on this one.