Openshift 4.3 Baremetal Installation - openshift

1. What I've tried
I want to make ocp cluster (actually, single node-all in one) like this blog
link : openshift.com/blog/revamped-openshift-all-in-one-aio-for-labs-and-fun
and I also referred to official document : Installing bare metal
So, What I have tried is like this :
(I used VirtualBox to make four vm)
- 1 bastion
- 1 dns
- 1 master
- 1 bootstrap
These vm are in the same network.
First, I made ignition file to boot master and bootstrap node.
install-config.yaml that I used :
apiVersion: v1
baseDomain: hololy-local.com
compute:
- hyperthreading: Enabled
name: worker
replicas: 0
controlPlane:
hyperthreading: Enabled
name: master
replicas: 1
metadata:
name: test
networking:
clusterNetwork:
- cidr: 10.128.0.0/14
hostPrefix: 23
networkType: OpenShiftSDN
serviceNetwork:
- 172.30.0.0/16
platform:
none: {}
fips: false
pullSecret: '{"auths": ...}'
sshKey: 'ssh-ed25519 AAAA...'
I only changed baseDomain, master's number of replica, pullSecret and sshKey.
After Making ignition files, I started to boot bootstrap node and master node with iso file.
bootstrap node was successfully installed, but problem happens master node.
2. Details
Before starting Master node installation, I have to set up dns. Because unlike bootstrap's installation, Master node requests domain info during installation.
ip address
dns : 192.168.56.114
master : 192.168.56.150
DNS Zone is like this :
And I started to set up master node using this parameters.
coreos.inst.install_dev=sda
coreos.inst.image_url=http://192.168.56.114/rhcos438.x86_64.raw.gz
coreos.inst.ignition_url=http://192.168.56.114/master.ign
ip=192.168.56.150::192.168.56.254:255.255.255.0:core0.hololy-local.com:enp0s3:none nameserver=192.168.56.114
Installation finished successfully, but when it boots without boot disk(.iso) Error comes out.
It seems to trying to find master configuration file in api-int.aio.hololy-local.com:22623, and It connects ip address that I wrote in the zone file.
But strangely, The connection refused continuously.
Since I set the static ip when rhcos installation, so Ping test works successfully to 192.168.56.150.
I think 22623 port was blocked. But How can I open the port before OS boot?...
I don't know how to I solve it.
Thanks.

I solved it.
The differences between installation of 3.11 and 4.x is whether LB's necessary.
In 4.x LB is necessary. so you should set up LB.
In my situation, I set LB by nginx, and the sample is like this:
stream{
upstream ocp_k8s_api {
#round-robin;
server 192.168.56.201:6443; #bootstrap
server 192.168.56.202:6443; #master1
server 192.168.56.203:6443; #master2
server 192.168.56.204:6443; #master3
}
server {
listen 6443;
proxy_pass ocp_k8s_api;
}
upstream ocp_m_config {
#round-robin;
server 192.168.56.201:22623; #bootstrap
server 192.168.56.202:22623; #master1
server 192.168.56.203:22623; #master2
server 192.168.56.204:22623; #master3
}
server {
listen 22623;
proxy_pass ocp_m_config;
}
upstream ocp_http {
#round-robin;
server 192.168.56.205:80; #worker1
server 192.168.56.206:80; #worker2
}
server{
listen 80;
proxy_pass ocp_http;
}
upstream ocp_https {
#round-robin;
server 192.168.56.205:443; #worker1
server 192.168.56.206:443; #worker2
}
server{
listen 443;
proxy_pass ocp_https;
}
}
thanks.

Related

Chrome ignores Nginx upstreams (loads only first)

I have simple setup of 3 servers (in containers) - 2 "app" servers (whoami services - so by response I can acknowledge server) and nginx server.
I've launched nginx with simple load-balancing configuration:
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
upstream myapp1 {
server w1:8000 weight=1;
server w2:8000 weight=1;
}
server {
listen 80;
location / {
proxy_pass http://myapp1/;
}
}
}
The problem is that it doesn't work in Chrome - it always loads only first server. I've tried to turn off cache in Dev console + reload via CTRL+F5 but nothing helped.
If I try to curl nginx server - I get responses in round robin manner (as expected).
Here is my containers setup:
docker network create testnw
docker run -dit --name w1 --network testnw jwilder/whoami # app1
docker run -dit --name w2 --network testnw jwilder/whoami # app2
docker run -dit --name ng --network testnw -p 8989:80 -v ${PWD}/my.conf:/etc/nginx/nginx.conf nginx # LB server
curl localhost:8989 # will get response from w1
curl localhost:8989 # will get response from w2
curl localhost:8989 # will get response from w1
...
Edit 3: Found out an interesting issue.
In chrome every time I access my website it makes two calls no matter what they are called to/of my website and /favicon.ico of my website.
I don't have a /favicon.ico.
What I think is happening
when Nginx is getting requests for/of my website, it is loading the first server upstream.
when chrome loads / from my website it also calls /favicon.ico of my website which results in making a new call to Nginx so it loads the .ico files from the next server upstream.
this happens so that servers 1,2,3 are loaded in order 1(ico file from 2),3(ico file from 1),2(ico file from 3). and cycle repeats.
once I stopped the loading of /favicon.ico in Nginx, my three upstreams servers 1,2,3 are loading in order 1,2,3 of round-robin.
I put this in the server with upstream to disable loading favicon.ico from Nginx.
location = /favicon.ico {
log_not_found off;
}
Hope anyone having this problem find this useful.
Edit 2: Figured out the issue, the load balancing is working fine with static files and static servers inside the Nginx conf file.
but my applications are being loaded by node, so had to start Nginx after starting all the node servers.
Issue reappears when I restart the application server while Nginx is running.
Now no issue will update soon
Edit 1: This is not working for me anymore, this worked yesterday, today continued working on the same configuration, the issue reappeared.
Had this same issue with my setup.
What worked for me after a lot of proxy setup and VirtualBox setup and network editing.
Add an extra server block in the HTTP block.
server{
}
and reload the Nginx service.
It worked for me, after reloading once both chrome and firefox loads the servers in the given order, I deleted the server block and it is still working.
Don't know why the issue raised in the first place.
Hope this helps to solve your issue.

route to application stopped working in OpenShift Online 3.9

I have an application running in Openshift Online starter, which worked for the last 5 months. A single pod behind a service with a route defined that does edge tls termination.
Since Saturday, when trying to access the application, I get the error message
Application is not available
The application is currently not serving requests at this endpoint. It may not have been started or is still starting.
Possible reasons you are seeing this page:
The host doesn't exist. Make sure the hostname was typed correctly and that a route matching this hostname exists.
The host exists, but doesn't have a matching path. Check if the URL path was typed correctly and that the route was created using the desired path.
Route and path matches, but all pods are down. Make sure that the resources exposed by this route (pods, services, deployment configs, etc) have at least one pod running.
The pod is running, I can exec into it and check this, I can port-forward to it and access it.
checking the different components with oc:
$ oc get po -o wide
NAME READY STATUS RESTARTS AGE IP NODE
taboo3-23-jt8l8 1/1 Running 0 1h 10.128.37.90 ip-172-31-30-113.ca-central-1.compute.internal
$ oc get svc
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
taboo3 172.30.238.44 <none> 8080/TCP 151d
$ oc describe svc taboo3
Name: taboo3
Namespace: sothawo
Labels: app=taboo3
Annotations: openshift.io/generated-by=OpenShiftWebConsole
Selector: deploymentconfig=taboo3
Type: ClusterIP
IP: 172.30.238.44
Port: 8080-tcp 8080/TCP
Endpoints: 10.128.37.90:8080
Session Affinity: None
Events: <none>
$ oc get route
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
taboo3 taboo3-sothawo.193b.starter-ca-central-1.openshiftapps.com taboo3 8080-tcp edge/Redirect None
I tried to add a new route as well (with or without tls), but am getting the same error.
Does anybody have an idea what might be causing this and how to fix it?
Addition April 17, 2018: Got an email from Openshift Online support:
It looks like you may be affected by this bug.
So waiting for it to be resolved.
The problem has been resolved by Openshift Online, the application is working again

go-ethereum - geth - puppeth - ethstat remote server : docker: command not found

I'm trying to setup a private ethereum test network using Puppeth (as Péter Szilágyi demoed in Ethereum devcon three 2017). I'm running it on a macbook pro (macOS Sierra).
When I try to setup the ethstat network component I get an "docker configured incorrectly: bash: docker: command not found" error. I have docker running and I can use it fine in the terminal e.g. docker ps.
Here are the steps I took:
What would you like to do? (default = stats)
1. Show network stats
2. Manage existing genesis
3. Track new remote server
4. Deploy network components
> 4
What would you like to deploy? (recommended order)
1. Ethstats - Network monitoring tool
2. Bootnode - Entry point of the network
3. Sealer - Full node minting new blocks
4. Wallet - Browser wallet for quick sends (todo)
5. Faucet - Crypto faucet to give away funds
6. Dashboard - Website listing above web-services
> 1
Which server do you want to interact with?
1. Connect another server
> 1
Please enter remote server's address:
> localhost
DEBUG[11-15|22:46:49] Attempting to establish SSH connection server=localhost
WARN [11-15|22:46:49] Bad SSH key, falling back to passwords path=/Users/xxx/.ssh/id_rsa err="ssh: cannot decode encrypted private keys"
The authenticity of host 'localhost:22 ([::1]:22)' can't be established.
SSH key fingerprint is xxx [MD5]
Are you sure you want to continue connecting (yes/no)? yes
What's the login password for xxx at localhost:22? (won't be echoed)
>
DEBUG[11-15|22:47:11] Verifying if docker is available server=localhost
ERROR[11-15|22:47:11] Server not ready for puppeth err="docker configured incorrectly: bash: docker: command not found\n"
Here are my questions:
Is there any documentation / tutorial describing how to setup this remote server properly. Or just on puppeth in general?
Can I not use localhost as "remote server address"
Any ideas on why the docker command is not found (it is installed and running and I can use it ok in the terminal).
Here is what I did.
For the docker you have to use the docker-compose binary. You can find it here.
Furthermore, you have to be sure that an ssh server is running on your localhost and that keys have been generated.
I didn't find any documentations for puppeth whatsoever.
I think I found the root cause to this problem. The SSH daemon is compiled with a default path. If you ssh to a machine with a specific command (other than a shell), you get that default path. This does not include /usr/local/bin for example, where docker lives in my case.
I found the solution here: https://serverfault.com/a/585075:
edit /etc/ssh/sshd_config and make sure it contains PermitUserEnvironment yes (you need to edit this with sudo)
create a file ~/.ssh/environment with the path that you want, in my case:
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
When you now run ssh localhost env you should see a PATH that matches whatever you put in ~/.ssh/environment.

How to make oc cluster up persistent?

I'm using "oc cluster up" to start my Openshift Origin environment. I can see, however, that once I shutdown the cluster my projects aren't persisted at restart. Is there a way to make them persistent ?
Thanks
There are a couple ways to do this. oc cluster up doesn't have a primary use case of persisting resources.
There are couple ways to do it:
Leverage capturing etcd as described in the oc cluster up README
There is a wrapper tool, that makes it easy to do this.
There is now an example in the cluster up --help command, it is bound to stay up to date so check that first
oc cluster up --help
...
Examples:
# Start OpenShift on a new docker machine named 'openshift'
oc cluster up --create-machine
# Start OpenShift using a specific public host name
oc cluster up --public-hostname=my.address.example.com
# Start OpenShift and preserve data and config between restarts
oc cluster up --host-data-dir=/mydata --use-existing-config
So specifically in v1.3.2 use --host-data-dir and --use-existing-config
Assuming you are using docker machine with vm such as virtual box, the easiest way I found is taking a vm snapshot WHILE vm and openshift cluster are up and running. This snapshot will backup memory in addition to disk therefore you can restore entire cluster later on by restoring the vm snapshot, then run docker-machine start ...
btw, as of latest os image openshift/origin:v3.6.0-rc.0 and oc cli, --host-data-dir=/mydata as suggested in the other answer doesn't work for me.
I'm using:
VirtualBox 5.1.26
Kubernetes v1.5.2+43a9be4
openshift v1.5.0+031cbe4
Didn't work for me using --host-data-dir (and others) :
oc cluster up --logging=true --metrics=true --docker-machine=openshift --use-existing-config=true --host-data-dir=/vm/data --host-config-dir=/vm/config --host-pv-dir=/vm/pv --host-volumes-dir=/vm/volumes
With output:
-- Checking OpenShift client ... OK
-- Checking Docker client ...
Starting Docker machine 'openshift'
Started Docker machine 'openshift'
-- Checking Docker version ...
WARNING: Cannot verify Docker version
-- Checking for existing OpenShift container ... OK
-- Checking for openshift/origin:v1.5.0 image ... OK
-- Checking Docker daemon configuration ... OK
-- Checking for available ports ... OK
-- Checking type of volume mount ...
Using Docker shared volumes for OpenShift volumes
-- Creating host directories ... OK
-- Finding server IP ...
Using docker-machine IP 192.168.99.100 as the host IP
Using 192.168.99.100 as the server IP
-- Starting OpenShift container ...
Starting OpenShift using container 'origin'
FAIL
Error: could not start OpenShift container "origin"
Details:
Last 10 lines of "origin" container log:
github.com/openshift/origin/vendor/github.com/coreos/pkg/capnslog.(*PackageLogger).Panicf(0xc4202a1600, 0x42b94c0, 0x1f, 0xc4214d9f08, 0x2, 0x2)
/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/coreos/pkg/capnslog/pkg_logger.go:75 +0x16a
github.com/openshift/origin/vendor/github.com/coreos/etcd/mvcc/backend.newBackend(0xc4209f84c0, 0x33, 0x5f5e100, 0x2710, 0xc4214d9fa8)
/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/coreos/etcd/mvcc/backend/backend.go:106 +0x341
github.com/openshift/origin/vendor/github.com/coreos/etcd/mvcc/backend.NewDefaultBackend(0xc4209f84c0, 0x33, 0x461e51, 0xc421471200)
/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/coreos/etcd/mvcc/backend/backend.go:100 +0x4d
github.com/openshift/origin/vendor/github.com/coreos/etcd/etcdserver.NewServer.func1(0xc4204bf640, 0xc4209f84c0, 0x33, 0xc421079a40)
/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/coreos/etcd/etcdserver/server.go:272 +0x39
created by github.com/openshift/origin/vendor/github.com/coreos/etcd/etcdserver.NewServer
/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/coreos/etcd/etcdserver/server.go:274 +0x345
Openshift writes to the directories /vm/... (also defined in VirtualBox) but successfully won't start.
See [https://github.com/openshift/origin/issues/12602][1]
Worked for me too, using Virtual Box Snapshots and restoring them.
To make it persistent after each shutdown you need to provide base-dir parameter.
$ mkdir ~/openshift-config
$ oc cluster up --base-dir=~/openshift-config
From help
$ oc cluster up --help
...
Options:
--base-dir='': Directory on Docker host for cluster up configuration
--enable=[*]: A list of components to enable. '*' enables all on-by-default components, 'foo' enables the component named 'foo', '-foo' disables the component named 'foo'.
--forward-ports=false: Use Docker port-forwarding to communicate with origin container. Requires 'socat' locally.
--http-proxy='': HTTP proxy to use for master and builds
--https-proxy='': HTTPS proxy to use for master and builds
--image='openshift/origin-${component}:${version}': Specify the images to use for OpenShift
--no-proxy=[]: List of hosts or subnets for which a proxy should not be used
--public-hostname='': Public hostname for OpenShift cluster
--routing-suffix='': Default suffix for server routes
--server-loglevel=0: Log level for OpenShift server
--skip-registry-check=false: Skip Docker daemon registry check
--write-config=false: Write the configuration files into host config dir
But you shouln't use it, because "cluster up" is removed in version 4.0.0. More here: https://github.com/openshift/origin/pull/21399

OpenShift Hazelcast

Is it possible to open a port for hazelcast on openshift? No matter what port I try, I get the same exception:
SocketException: Permission denied
I am not trying to open the port to the world. I just want to open a port so the gears can use Hazelcast. It seems like this should be possible.
You're probably have to use a HTTP tunnel to connect Hazelcast, not a nice solution but I prototyped it some time ago: https://github.com/noctarius/https-tunnel-openshift-hazelcast
Anyhow gears should be openshift V2, isn't it? Never tried it with V2, if you get the chance, there's support for V3 (and V3.1) - http://blog.hazelcast.com/openshift/
What cartridge type do you use?
You can bind to any port from 15000 to 35530 internally, but other gears won't be able to access it.
From my experience - I had to open the public proxy port for other members of the cluster to join.
For example, Vert.x cartridge uses Hazelcast for clustering and has some additional public proxy ports open (see https://github.com/vert-x/openshift-cartridge/blob/master/metadata/manifest.yml).
Endpoints:
- Private-IP-Name: IP
Private-Port-Name: PORT
Private-Port: 8080
Public-Port-Name: PROXY_PORT
Mappings:
- Frontend: ""
Backend: ""
Options: { "websocket": 1}
- Private-IP-Name: IP
Private-Port-Name: HAZELCAST_PORT
Private-Port: 5701
Public-Port-Name: HAZELCAST_PROXY_PORT
- Private-IP-Name: IP
Private-Port-Name: CLUSTER_PORT
Private-Port: 9123
Public-Port-Name: CLUSTER_PROXY_PORT
(see https://access.redhat.com/documentation/en-US/OpenShift_Online/2.0/html/Cartridge_Specification_Guide/chap-Exposing_Services.html).
On OpenShift, you should only bind websockets to either port 8000 or 8443.
See:
https://developers.openshift.com/en/managing-port-binding-routing.html
https://blog.openshift.com/paas-websockets/