SSH to Google Compute instance using NodeJS, without gcloud - google-compute-engine

I'm trying to create a SSH tunnel into a compute instance, from an environment that doesn't have gcloud installed (App Engine Standard NodeJS Environment).
What are the steps needed to do that? How does gcloud compute ssh command does it? Is there a NodeJS library that already does it?

I created the package gcloud-ssh-tunnel that does the necessary steps:
Create a private/public key using sshpk
Imports the public key using the OS Login API
SSH using ssh2 (and specifically create a tunnel, because this was the use case I needed - see the Why? section in the package)
Delete the public key using the OS Login API (to not overflow the account or leave security access)

You can use ssh2 to do that in nodejs.

"gcloud compute ssh" generates persistent SSH keys for the user. The public key is stored in project or instance SSH keys metadata, and the Guest Environment creates the necessary local user and places ~/.ssh/authorized_keys in its home directory.
You can manually add your public key to the instance, and then connect to it via ssh using a node ssh library1.
Or you can set a startup script for the instance when you are creating it2.
As Cloud Ace pointed out, you can use the ssh2 module3 for node.js compatibility.

In order to SSH into a GCP instance you have to:
Enable OS Login
Create a service account and assign it "Compute OS Admin Login" role.
Create SSH key and import it into the service account.
Use that SSH key and POSIX username.
The first 2 steps already link to the documentation.
Create SSH key:
import {
generatePrivateKey,
} from 'sshpk';
const keyPair = generatePrivateKey('ecdsa');
const privateKey = keyPair.toString();
const publicKey = keyPair.toPublic().toString();
Import key:
const osLoginServiceClient = new OsLoginServiceClient({
credentials: googleCredentials,
});
const [result] = await osLoginServiceClient.importSshPublicKey({
parent: osLoginServiceClient.userPath(googleCredentials.client_email),
sshPublicKey: {
expirationTimeUsec: ((Date.now() + 10 * 60 * 1_000) * 1_000).toString(),
key: publicKey,
},
});
SSH using the key:
const ssh = new NodeSSH();
await ssh.connect({
host,
privateKey,
username: loginProfile.posixAccounts[0].username,
});
In this example, I am using node-ssh but you can use anything.
The only other catch is that you need to figure out the public host. Implementation for that looks like this:
const findFirstPublicIp = async (
googleCredentials: GoogleCredentials,
googleZone: string,
googleProjectId: string,
instanceName: string,
) => {
const instancesClient = new InstancesClient({
credentials: googleCredentials,
});
const instances = await instancesClient.get({
instance: instanceName,
project: googleProjectId,
zone: googleZone,
});
for (const instance of instances) {
if (!instance || !('networkInterfaces' in instance) || !instance.networkInterfaces) {
throw new Error('Unexpected result.');
}
for (const networkInterface of instance.networkInterfaces) {
if (!networkInterface || !('accessConfigs' in networkInterface) || !networkInterface.accessConfigs) {
throw new Error('Unexpected result.');
}
for (const accessConfig of networkInterface.accessConfigs) {
if (accessConfig.natIP) {
return accessConfig.natIP;
}
}
}
}
throw new Error('Could not locate public instance IP address.');
};
Finally, to clean up, you have to call deleteSshPublicKey with the name of the key that you've imported:
const fingerprint = crypto
.createHash('sha256')
.update(publicKey)
.digest('hex');
const sshPublicKey = loginProfile.sshPublicKeys?.[fingerprint];
if (!sshPublicKey) {
throw new Error('Could not locate SSH public key with a matching fingerprint.');
}
const ssh = new NodeSSH();
await ssh.connect({
host,
privateKey,
username: loginProfile.posixAccounts[0].username,
});
await osLoginServiceClient.deleteSshPublicKey({
name: sshPublicKey.name,
});

In general, you'd need to reserve & assign a static external IP address to begin with (unless trying to SSH from within the same network). And a firewall rule needs to be defined for port tcp/22, which then can be applied as a "label" to the network interface, which has that external IP assigned.
The other way around works with gcloud app instances ssh:
SSH into the VM of an App Engine Flexible instance
which might be less effort & cost to setup, because a GCP VM usually has gcloud installed.

Related

Create a Network Load Balancer on Oracle Cloud Infrastructure with a Reserved IP using Terraform

Using Terraform to set up a Network Load Balancer on Oracle Cloud Infrastructure, it works as expected if created with an ephemeral public IP, however one created using a reserved public IP does not respond. Here are the exact Terraform resourses used to create the load balancer:
resource "oci_core_public_ip" "ip" {
for_each = { for lb in var.load_balancers: lb.subnet => lb if ! lb.private
compartment_id = local.compartment_ocid
display_name = "${var.name}-public-ip"
lifetime = "RESERVED"
lifecycle {
prevent_destroy = true
}
}
resource "oci_network_load_balancer_network_load_balancer" "nlb" {
for_each = { for lb in var.load_balancers: lb.subnet => lb if lb.type == "network" }
compartment_id = local.compartment_ocid
display_name = "${var.name}-network-load-balancer"
subnet_id = oci_core_subnet.s[each.value.subnet].id
is_private = each.value.private
#reserved_ips {
# id = oci_core_public_ip.ip[each.value.subnet].id
#}
}
All of the other resources: security list rules, listeners, backend set and backends, etc, etc, are created such that the above works. If, however I uncomment the assignment of reserved_ips to the network load balancer then it does not work: no response from the load balancer's public IP. Everything is the same except those three lines being uncommented.
Between each test I tear down everything and recreate with Terraform. It always works with an ephemeral IP and never works with the reserved IP. Why? What am I missing? Or does this just not work as advertised?
The Terraform version is v1.3.4 and the resource version is oracle/oci version 4.98.0.
The reserved IP is set up correctly however the terraform provider removes its association with the load balancer's private IP. Closer inspection of the Terraform output shows this
~ resource "oci_core_public_ip" "ip" {
id = "ocid1.publicip.oc1.uk-london-1.ama...sta"
- private_ip_id = "ocid1.privateip.oc1.uk-london-1.abw...kya" -> null
# (11 unchanged attributes hidden)
}
Manually replacing it fixes it (until the next tf run)
$ oci network public-ip update --public-ip-id ocid1.publicip.oc1.uk-london-1.ama...rrq --private-ip-id ocid1.privateip.oc1.uk-london-1.abw...kya
There is a bug ticket on Terraform's github.

aws cdk ecs task scheduling specify existing securitygroup

When defining an ECS Task Schedule, I can't seem to find a way of specifying an existing security group. Any pointers on where this can be configured using aws cdk?
In the code snippet below, you'll see I am able to create a cron, specify the docker image to schedule and create the schedule itself by specifying the existing cluster and vpc. However, there is no option to specify an existing security group... Is it possible to specify an existing security group?
schedule_cron = scaling.Schedule.cron(minute=manifest['schedule']['minute'],
hour=manifest['schedule']['hour'],
day=manifest['schedule']['day'],
month=manifest['schedule']['month'],
year=manifest['schedule']['year'])
image_option = ecs_patterns.ScheduledFargateTaskImageOptions(image=img,
cpu=manifest["resources"]["cpu"],
memory_limit_mib=manifest["resources"]["memory"],
log_driver=ecs.AwsLogDriver(log_group=log_group,
stream_prefix=manifest["app_name"]),
secrets=secrets,
environment= env)
schedule_pattern = ecs_patterns.ScheduledFargateTask(self, f"scheduledtask{app_name}",
schedule= schedule_cron, scheduled_fargate_task_image_options=image_option, cluster=cluster,
desired_task_count=manifest["replica_count"], vpc=vpc)
The ECS Patterns does not support this yet. The underlying constructs however do. Therefore you must specify the TaskDefinition, Event and Event Target yourself.. With Event the schedule is specified and with Event Target the SecurityGroup is set.
Here is an example implementation using TypeScript. Please adjust this to Python using the aws_cdk.aws_events and aws_cdk.aws_events_targets modules.
import aas = require('#aws-cdk/aws-applicationautoscaling');
import cdk = require('#aws-cdk/core');
import events = require("#aws-cdk/aws-events")
import event_targets = require("#aws-cdk/aws-events-targets");
import ec2 = require('#aws-cdk/aws-ec2');
const securityGroup = new ec2.SecurityGroup(this, "SecurityGroup", {
vpc: vpc,
});
const task = new ecs.FargateTaskDefinition(this, "TaskDefinition", {
family: "ScheduledTask",
cpu: ..,
memoryLimitMiB: ..,
});
task.addContainer("app_name", ...);
const rule = new events.Rule(this, "Rule", {
description: "ScheduledTask app_name Trigger",
enabled: true,
schedule: aas.Schedule.rate(cdk.Duration.hours(1)),
targets: [
new event_targets.EcsTask({
cluster: cluster,
taskDefinition: task,
securityGroup: securityGroup,
}),
],
});
Please note that the EcsTask event target only allows one security group. This issue was raised a while ago on GitHub: https://github.com/aws/aws-cdk/issues/3312

How to find CPU MEMORY usage with docker stats command?

I am using docker-java API to execute docker API in my project. I didn't find any suitable method which lists down docker CPU memory usage as
GET /v1.24/containers/redis1/stats HTTP/1.1 with the help of docker-java API
Dependency
compile group: 'com.github.docker-java', name: 'docker-java', version: '3.1.2'
Code
public static void execute() {
DockerClient dockerClient = DockerClientBuilder.getInstance().build();
dockerClient.statsCmd("containerName");
}
I didn't get any output
Tell me how to execute docker stats with docker-java api
This works for me
public Statistics getNextStatistics() throws ProfilingException {
AsyncResultCallback<Statistics> callback = new AsyncResultCallback<>();
client.statsCmd(containerId).exec(callback);
Statistics stats;
try {
stats = callback.awaitResult();
callback.close();
} catch (RuntimeException | IOException e) {
// you may want to throw an exception here
}
return stats; // this may be null or invalid if the container has terminated
}
DockerClient is where we can establish a connection between a Docker engine/daemon and our application.
By default, the Docker daemon can only be accessible at the unix:///var/run/docker.sock file. We can locally communicate with the Docker engine listening on the Unix socket unless otherwise configured.
we can open a connection in two steps:
DefaultDockerClientConfig.Builder config
= DefaultDockerClientConfig.createDefaultConfigBuilder();
DockerClient dockerClient = DockerClientBuilder
.getInstance(config)
.build();
Since engines could rely on other characteristics, the client is also configurable with different conditions.
For example, the builder accepts a server URL, that is, we can update the connection value if the engine is available on port 2375:
DockerClient dockerClient
= DockerClientBuilder.getInstance("tcp://docker.baeldung.com:2375").build();
Note that we need to prepend the connection string with unix:// or tcp:// depending on the connection type.

Bare Metal Cloud - How to set authorized ssh keys for compute instances?

I have successfully provisioned Bare Metal Cloud compute instances using the following code:
public static Instance createInstance(
ComputeClient computeClient,
String compartmentId,
AvailabilityDomain availabilityDomain,
String instanceName,
Image image,
Shape shape,
Subnet subnet
) {
LaunchInstanceResponse response = computeClient.launchInstance(
LaunchInstanceRequest.builder()
.launchInstanceDetails(
LaunchInstanceDetails.builder()
.availabilityDomain(availabilityDomain.getName())
.compartmentId(compartmentId)
.displayName(instanceName)
.imageId(image.getId())
.shape(shape.getShape())
.subnetId(subnet.getId())
.build())
.build());
return response.getInstance();
}
However, I can't SSH into any instances I create via the code above, because there's no parameter on launchInstance to pass in the public key of my SSH keypair.
How can I tell the instance what SSH public key to allow? I know it must be possible somehow since the console UI allows me to provide the SSH public key as part of instance creation.
According to the launch instance API documentation, you need to pass your SSH public key via the ssh_authorized_keys field of the metadata parameter:
Providing Cloud-Init Metadata
You can use the following metadata key names to provide information to Cloud-Init:
"ssh_authorized_keys" - Provide one or more public SSH keys to be
included in the ~/.ssh/authorized_keys file for the default user on
the instance. Use a newline character to separate multiple keys. The
SSH keys must be in the format necessary for the authorized_keys file
The code for this in the Java SDK looks like this:
public static Instance createInstance(
ComputeClient computeClient,
String compartmentId,
AvailabilityDomain availabilityDomain,
String instanceName,
Image image,
Shape shape,
Subnet subnet
) {
String sshPublicKey = "ssh-rsa AAAAB3NzaC1y...key shortened for example...fdK/ABqxgH7sy3AWgBjfj some description";
Map<String, String> metadata = new HashMap<>();
metadata.put("ssh_authorized_keys", sshPublicKey);
LaunchInstanceResponse response = computeClient.launchInstance(
LaunchInstanceRequest.builder()
.launchInstanceDetails(
LaunchInstanceDetails.builder()
.availabilityDomain(availabilityDomain.getName())
.compartmentId(compartmentId)
.displayName(instanceName)
.imageId(image.getId())
.metadata(metadata)
.shape(shape.getShape())
.subnetId(subnet.getId())
.build())
.build());
return response.getInstance();
}
Then the instance will allow you to SSH to it using the SSH keypair for that public key.

What's the best way to migrate to ServiceStack authentication framework when stuck with my_aspnet_* tables

I'm not quite ready to change up all my user/auth tables from the MySQL user/roles/profile provider format, but am moving off of MVC to ServiceStack.
Is there a pre-built IUserAuthRespository and/or CredentialsAuthProvider somewhere that can be used, or do I need to build one to provide this mapping?
If I need to build one, I assume implementing at the IUserAuthRepository level is the cleanest? Is there a minimum set of methods required to implement basic login/logout (and administrative "switch user" impersonation) functionality?
I tried implementing a custom CredentialsAuthProvider, which seems to work, but I'm unable to get local posts for impersonation to use the proper provider. Looking for a solution to that, I realized that maybe its better to implement the repository instead.
EDIT:
My current registration of the custom auth provider is:
Plugins.Add(new AuthFeature(() => new AuthUserSession(), new IAuthProvider[]
{
container.Resolve<MySqlCredentialsAuthProvider>() //HTML Form post of UserName/Password credentials
}));
And calling code for the local post to the AuthenticateService is:
[RequiredRole(SystemRoles.Administrator)]
public object Any(ImpersonateUser request)
{
using (var service = base.ResolveService<AuthenticateService>()) //In Process
{
//lets us login without a password if we call it internally
var result = service.Post(new Authenticate
{
provider = AuthenticateService.CredentialsProvider,
UserName = request.Username,
//Password = "should-not-matter-since-we-are-posting-locally"
});
return result;
}
}
Integrating with existing User Auth tables
If you want to use your existing User/Auth tables, the easiest solution is to ignore the UserAuth repositories and implement a Custom CredentialsAuthProvider that looks at your existing database tables to return whether their Authentication attempt was successful.
Implement OnAuthenticated() to populate the rest of your typed IAuthSession from your database, e.g:
public class CustomCredentialsAuthProvider : CredentialsAuthProvider
{
public override bool TryAuthenticate(IServiceBase authService,
string userName, string password)
{
//Add here your custom auth logic (database calls etc)
//Return true if credentials are valid, otherwise false
}
public override IHttpResult OnAuthenticated(IServiceBase authService,
IAuthSession session, IAuthTokens tokens,
Dictionary<string, string> authInfo)
{
//Fill IAuthSession with data you want to retrieve in the app eg:
session.FirstName = "some_firstname_from_db";
//...
//Call base method to Save Session and fire Auth/Session callbacks:
return base.OnAuthenticated(authService, session, tokens, authInfo);
//Alternatively avoid built-in behavior and explicitly save session with
//authService.SaveSession(session, SessionExpiry);
//return null;
}
}
Importing existing User Auth tables
If you want to import them into an OrmLite User Auth tables, you would configure to use the OrmLiteAuthRepository in your AppHost:
//Register to use MySql Dialect Provider
container.Register<IDbConnectionFactory>(
new OrmLiteConnectionFactory(dbConnString, MySqlDialect.Provider));
Plugins.Add(new AuthFeature(
() => new CustomUserSession(), //Use your own typed Custom UserSession type
new IAuthProvider[] {
//HTML Form post of UserName/Password credentials
new CredentialsAuthProvider()
}));
//Tell ServiceStack you want to persist User Info in the registered MySql DB above
container.Register<IUserAuthRepository>(c =>
new OrmLiteAuthRepository(c.Resolve<IDbConnectionFactory>()));
//Resolve instance of configured IUserAuthRepository
var userAuth = container.Resolve<IUserAuthRepository>();
//Create any missing UserAuth RDBMS tables
authRepo.InitSchema();
Then to import your data you can use the above MySQL DB connection to select from your existing tables then use the IUserAuthRepository to create new Users.
// Open DB Connection to RDBMS
using (var db = container.Resolve<IDbConnectionFactory>().Open())
{
//Example of fetching old Users out of a custom table (use your table instead)
var oldUsers = db.Select<OldUserInfo>();
// Clear existing UserAuth tables if you want to replay this import
//db.DeleteAll<UserAuthDetails>();
//db.DeleteAll<UserAuth>();
//Go through and create new User Accounts using Old User Info
foreach (var oldUser in oldUsers)
{
//Create New User Info from Old Info
var newUser = new UserAuth {
UserName = oldUser.UserName,
Email = oldUser.Email,
//...
};
//Create New User Account with oldUser Password
authRepo.CreateUserAuth(newUser, oldUser.Password);
}
}
After this you'll have new User Accounts from your old User Info which you can sign in with.