SSH not working with password after upgrade ubuntu 22.04 or above

In recent upgrade of ubuntu 22.04 and above we are not able to login server with SSH password. but when we try to login with key then it allows to login.

Reason:

The RSA algorithm keys we are using till now it no longer supported. the resolution is to use new keys generated by a more secure algorithm, such as ed25519 or disable the one configuration in ssh configuration file.

Resolution:

I have done by commenting one line in ssh configuration.

1. open ssh configuration file,

/etc/ssh/sshd_config

2. comment below line,

#KbdInteractiveAuthentication no

3. Restart ssh service,

service ssh restart

 and now its working.

Gitlab community version upgrade from 11.8.0 to 16.7.0

 Gitlab community version upgrade from 11.8.0 to 16.7.0

Note: You can't upgrade gitlab-ce version directly from old to new as there is some database migrations in between the minor releases.

First upgrade the latest minor version of current majot version. i.e. 11.11.0

Please follow the steps below:

1. 11.8.0 -> 11.11.0

Download link:

https://packages.gitlab.com/gitlab/gitlab-ce/packages/debian/bookworm/gitlab-ce_11.11.0-ce.0_amd64.deb

2. 11.11.0 -> 12.0.0

3. 12.0.0 -> 12.10.14

4. 13.0.0 -> 13.12.15

5. 14.0.0 -> 14.1.1 -> 14.5.4 -> 14.9.5 -> 14.10.5

14.0.0 Installation:

If Error:

To skip legacy storage data migration,

gitlab preinstall: Checking for unmigrated data on legacy storage

gitlab preinstall:

gitlab preinstall: Legacy storage is no longer supported. Please migrate your data to hashed storage.

gitlab preinstall: Check https://docs.gitlab.com/ee/administration/raketasks/storage.html#migrate-to-hashed-storage for details.

gitlab preinstall:

gitlab preinstall: If you want to skip this check, run the following command and try again:

gitlab preinstall:

gitlab preinstall:  sudo touch /etc/gitlab/skip-unmigrated-data-check

gitlab preinstall:

dpkg: error processing archive gitlab-ce_14.0.0-ce.0_amd64.deb (--install):

Solution:

root@Gitlab-11-8:~# gitlab-rails dbconsole

psql (12.6)

Type "help" for help.

gitlabhq_production=> UPDATE projects SET runners_token = null, runners_token_encrypted = null;

UPDATE 41

gitlabhq_production=> UPDATE namespaces SET runners_token = null, runners_token_encrypted = null;

UPDATE 11

gitlabhq_production=> UPDATE application_settings SET runners_registration_token_encrypted = null;

UPDATE 1

gitlabhq_production=> UPDATE ci_runners  SET token = null, token_encrypted = null;

UPDATE 0

gitlabhq_production=> exit

root@Gitlab-11-8:~# gitlab-rake gitlab:storage:migrate_to_hashed

And try to install it again.

Then install 14.1.1, 14.5.4.

If Error while migrating database then try below commands,

$ sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,events,id,'[["id"]\, ["id_convert_to_bigint"]]']

$ sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,ci_builds_metadata,id,'[["id"]\, ["id_convert_to_bigint"]]']

$ gitlab-ctl reconfigure

Then install 14.9.5, 14.10.5.

If got below error and you stuck up on gitlab-ctl reconfigure then,

rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash[migrate gitlab-rails database] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb line 16) had an error: Mixlib::ShellOut::ShellCommandFailed: Command execution failed. STDOUT/STDERR suppressed for sensitive resource

Solution:

sudo gitlab-ctl restart postgresql

Change in below file to skip migration,

nano /etc/gitlab/gitlab.rb

gitlab_rails['auto_migrate'] = false

Run below command,

$ gitlab-ctl reconfigure

Once reverted to old version, revert the changes in gitlab.rb and then resolve the errors and try to upgrade it again.

6. 14.10.5 -> 15.0.5 -> 15.1.6 -> 15.11.13

7. 16.0.8 -> 16.1.5 -> 16.2.8 -> 16.3.6 -> 16.7.0

Done.

SSL setup in webservers

 

1.  Setup SSL in tomcat with using Certificate, Chain certificate and Private key.

for example,

1. website.crt ( main certificate file )

2. website-ca.crt ( chain certificate )

3. website.key ( private Key )

Open server.xml which is located in tomcat base directory.

nano $CATALINA_BASE/Tomcat/conf/server.xml

<Connector port="443" protocol="org.apache.coyote.http11.Http11AprProtocol"
                compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json" 
                compression="force" compressionMinSize="1024" useSendfile="false" noCompressionUserAgents="gozilla, traviata"   
                URIEncoding="UTF-8" useBodyEncodingForURI="true"
                connectionTimeout="20000"
                enableLookups="false" 
                maxThreads="1500" SSLEnabled="true" 
                defaultSSLHostConfigName="Primary-Domain-name" maxParameterCount="-1">

                <SSLHostConfig hostName="Domain-name" protocols="TLSv1.2">
                              <Certificate certificateFile="PATH-To-SSL-Directory/website.crt"
                                        certificateKeyFile="PATH-To-SSL-Directory/website.key"
                                        certificateChainFile="PATH-To-SSL-Directory/website-ca.crt" />
                </SSLHostConfig>                                                 </Connector>

Restart Tomcat service.

2. Generate private key from Certificate and Chain certificate.

Generate CSR:

keytool -keysize 2048 -genkey -alias website -keyalg RSA -keystore website.keystore

keytool -certreq -keyalg RSA -alias website -file website.csr -keystore website.keystore

[Keep the password used at the time of generating CSR.]

Generate Certificates using Generated CSR from respected domain provider.

keytool -import -alias root -keystore website.keystore -trustcacerts -file website.crt

keytool -import -alias intermed -keystore website.keystore -trustcacerts -file website-ca.crt

keytool -import -alias website -keystore website.keystore -trustcacerts -file website.crt

keytool -importkeystore -srckeystore website.keystore -destkeystore website.keystore -deststoretype pkcs12

openssl pkcs12 -in website.keystore -out website.pem

Now add configuration into server.xml,

nano $CATALINA_BASE/Tomcat/conf/server.xml

<Connector port="443" protocol="org.apache.coyote.http11.Http11AprProtocol"
                compressableMimeType="text/html,text/xml,text/plain,text/css,text/javascript,application/javascript,application/json" 
                compression="force" compressionMinSize="1024" useSendfile="false" noCompressionUserAgents="gozilla, traviata"   
                URIEncoding="UTF-8" useBodyEncodingForURI="true"
                connectionTimeout="20000"
                enableLookups="false" 
                maxThreads="1500" SSLEnabled="true" 
                defaultSSLHostConfigName="Primary-Domain-name" maxParameterCount="-1">

                 <SSLHostConfig hostName="Domain-name">
                         <Certificate certificateFile="PATH-TO-SSL/website.pem" certificateKeyPassword="certificate-password"
                                      certificateChainFile="PATH-TO-SSL/website-ca.crt" />
                </SSLHostConfig>

</Connector> 

3. Setup Above SSL certificates on Nginx webserver.

below are the files which is available which we have generated earlier.[ in 2nd solution ]

1. website.csr

2. website.crt

3. website-ca.crt

4. website.pem

a. convert generated .pem file into key file using below command.

openssl rsa -text -in website.pem

Copy the generated context into one file named website.key. example of key file is given below.

-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAgVa6C3+/p3AfEn1ZW8KgnxAegMgjv/AllpBEeC0lB2gAkecS
xvraI+PyyzjI7maEktIPw4k2TXafsl/OYZXbPOXpSVaeIZyKCW80RsApDvJysJq/
V0g6WQKBgQCzJMUlOXFUrPKelDfhS4/7YNYmm2+ztcj/X7SXMvaA7vY424KHc9br
MzqnExB01Ge1VDB5xKrEeWtcp30mQ8E9KQfVEIlJO4dsCDVL32+beY57kWbIivlC
mf3YvqeBdnRRSjanxn5bsHqu91GLP4jQIuBNKkz8dRYwBSVcZnb1SA==
-----END RSA PRIVATE KEY-----

b. create combine certificate file by combining CRT and Chain certificate file.

cat website.crt > website-combine.crt
cat website-ca.crt > website-combine.crt

c. open SSL configuration file of nginx.

/etc/nginx/conf.d/ssl.conf

server {
    listen      443 ssl http2;
    listen      [::]:443 ssl http2;
    server_name domainname.in;
    ssl_certificate /$PATH-TO-SSL/website.crt;
    ssl_certificate_key /$PATH-TO-SSL/website.key;;
    # SSL Settings
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;
    ssl_session_cache   shared:SSL:20m;
    ssl_session_timeout 4h;
    ssl_session_tickets off;
   # OCSP stapling
    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_trusted_certificate /$PATH-TO-SSL/website-combine.crt;
        location / {
                proxy_pass              https://tomssl/;
                proxy_set_header        Host               $host;
                proxy_set_header        X-Real-IP          $remote_addr;
                proxy_set_header        X-Forwarded-For    $remote_addr;
                proxy_set_header        X-Forwarded-Host   $host:443;
                proxy_set_header        X-Forwarded-Server $host;
                proxy_set_header        X-Forwarded-Port   443;
                proxy_set_header        X-Forwarded-Proto  $scheme;
                allow all;
        }
        error_page 500 502 503 504 /server-busy.html;
        location = /server-busy.html {
                root /usr/share/nginx/html;
                internal;
        }
}

d. Restart Nginx.

nginx -t
systemctl restart nginx

4. Setup SSL in apache webserver.

Open SSL configuration file of apache. and add below configuration.

/etc/apache2/sites-available/default-ssl.conf

        <VirtualHost *:443>
                <Directory /path/to/website/dir>
                    Options Indexes FollowSymLinks MultiViews
                    AllowOverride All
                    Require all granted
                </Directory>
                ServerAdmin purval@test.com
                ServerName impurval.blogspot.com
                ServerAlias www.impurval.blogspot.com
                DocumentRoot /path/to/website/dir
                ErrorLog ${APACHE_LOG_DIR}/website.log
                #CustomLog ${APACHE_LOG_DIR}/phpmyadmin_access.log combined
                SSLEngine on
                SSLCertificateFile "/path/to/ssl/website.crt"
                SSLCertificateKeyFile "/path/to/ssl/website.key"
                SSLCertificateChainFile "path/to/ssl/website-ca.crt"
                <FilesMatch "\.(cgi|shtml|phtml|php)$">
                        SSLOptions +StdEnvVars
                </FilesMatch>
                <Directory /usr/lib/cgi-bin>
                        SSLOptions +StdEnvVars
                </Directory>
                Protocols h2 h2c http/1.1
        </VirtualHost>

Restart apache.

apachectl -t
/etc/init.d/apache2 restart

5. Setup free Lets encrypt SSL.

Please note that this SSL certificates are available fro 3 months only then we have to renew it.

1. Install certbot package if not installed.

apt-get install certbot

2. If webserver is already running on port 443 with SSL then need to stop application first.

3. Generate certificates using below command.

sudo certbot certonly --standalone -d domain-name.com

4. Add the configuration entry into configuration file. below is the configuration settings of SSL for tomcat webserver.

nano $CATALINA_BASE/config/server.xml

<SSLHostConfig hostName="domain-name.com">
                	<Certificate certificateFile="/etc/letsencrypt/live/domain-name.com/cert.pem"
                   	certificateKeyFile="/etc/letsencrypt/live/domain-name.com/privkey.pem"
                   	certificateChainFile="/etc/letsencrypt/live/domain-name.com/chain.pem" />
</SSLHostConfig>

Block specifc web access call using nginx

 Task:

Block specific web calls (e.g. /webservice?token=test123) and allow other calls using nginx.

Solution:

1. make changes in below nginx configuration file.

nano /etc/nginx/conf/80.conf

upstream backend {
    ip_hash;
    server 172.31.25.94;
}
server {
    listen 80;
    listen      [::]:80;
        ## Block  test123 call        
        location /webservice {
            if ($arg_token = "test123") {
                return 403;
            }
            proxy_pass  http://backend;
        }
        ## Block  test123 call        
        location / {
                proxy_pass  http://backend;
                proxy_set_header        Host              $http_host;
                proxy_set_header        X-Forwarded-By    $server_addr:$server_port;
                proxy_set_header        X-Forwarded-For   $remote_addr;
                proxy_set_header        X-Forwarded-Proto $scheme;
                proxy_set_header        X-Real-IP         $remote_addr;
        }
}

Same thing need to do this in ssl.conf file as well.

2. verify nginx  configuration.

nginx -t

3. reload nginx service.

systemctl reload nginx