VM Config
All Distributions
VM Specifications:
- CPU: 1vCPU (2 cores)
- RAM: 2 GB
- HD: 100 GB
Rocky
Change the hostname:
Rocky 8 - Verify and/or Update the IP
Rocky 9 - Verify and/or Update the IP
Verify DNS is setup correctly.
The output should look something like:
Verify NTP is setup correctly.
The output should look something like:
Ubuntu
Change the hostname:
Verify and/or Update the IP
No DNS changes are required. Ubuntu uses a dynamic resolv.conf file for connecting local clients to the internal DNS stub resolver of systemd-resolved.
Verify NTP is setup correctly.
The output should look something like:
Install
All Distributions
Package Descriptions
- adcli: tools for joining and managing AD domains
- oddjob-mkhomedir: creates home dir for AD accounts
- oddjob: a D-bus service that runs odd jobs for clients
- realmd: This manages enrolment and membership to AD
- samba-common: shared tools for servers and clients
- samba: This denotes the Samba services
- sssd: used to divert client authentication as required
Rocky
Update the system:
Install the pre-reqs:
Ubuntu
In the sanatize guide, we used the apt-get command. This is the older standard command which is a subset of apt. Going forward, we will use the "apt" command instead of "apt-get.
Update the system:
Install the pre-reqs:
SSL Chain Install
All Distributions
The PEM format is the most common format used for certificates. Extensions used for PEM certificates are: cer, crt, and pem. They are Base64 encoded ASCII files that contain human readable content between --BEGIN CERTIFICATE-- and --END CERTIFICATE--. The DER format is the binary form of the certificate. DER formatted certificates do not contain the "BEGIN CERTIFICATE/END CERTIFICATE" statements. DER formatted certificates most often use the '.der' extension.
The certs should have already been converted from the a "base-64 encoded x.509 (.cer)" Windows encoding to a Unix encoding. If this has not already happened:
Run the following OpenSSL commands:
Combine the files (if needed):
Download the certs to the local system. In the respective distro cert install commands, adding a combined chain would be redundant. So we only use the root and intermediate, not cert chain.
Set permissions
Change ownsership
Certificate Creation
Certificate creation is out of scope of this document.
SSL Cert Install
Rocky
Move the files to their home for the import.
Install the certs.
After you run the "update-ca-trust" command, the certs are moved to: /etc/pki/ca-trust/extracted. Applications that look to this directory to verify certificates can use any of the formats provided. The update command handles the copies, conversions, and consolidation for the different formats.
To verify if the cert installed successfully:
Test the cert and AD connectivity:
Ubuntu
Move the files to their home for the import. Make sure that the filename ends with .crt.
Install the certs.
The expected output from the update-ca-certificates command is: "3 added, 0 removed; done."
To verify if the cert installed successfully:
Test the cert and AD connectivity:
Join to Active Directory
All Distributions
Verify the machine is not already part of AD.
Test connectivity to Active Directory.
To set the OS info and AD OU:
If you are using a distro or version other than Rocky 8, update the OS information for Active Directory.
Rocky 9
Ubuntu 20.04
If you are joing a machine other than a server to Active Directory, correct the OU where the object is to reside.
Restart realmd.
Join to the domain.
After AD join, /etc/sssd/sssd.conf, /etc/krb.conf, and /etc/krb5.keytab will be automatically created. If you get "realm: Couldn't join realm: Joining the domain domain.local failed", restart realmd and retry. If this fails, add -v for verbose to the end of the command.
To verify the system was successfully joined to AD:
Verify that you can perform a AD user lookup. The IDs are the numbers that will need to be changed in Active Directory. See the "AD Server - Configure NFS ID Mapping" section for more.
Test the AD connectivity:
SSSD Config
All Distributions
SSSD stands for: System Security Services Daemon.
Make a backup of the respective conf file.
The differances between the Rocky 8 and Ubuntu 20.04 sssd.conf file were minimal. We opted to combine the required settings into a single config file. We are clearing the config file and rewriting it.
Be aware that if you are using the AD provider, your communication across port 389 is encrypted using GSSAPI (Kerberos). It uses the host keytab to encrypt that communication. Using SSL atop that would be a waste of resources (and unsupported by Microsoft).
If you have GSSAPI encryption available (you do) then SSSD ignores the "ldap_id_use_start_tls" argument because you don't need both encryption streams. `ldap_id_use_start_tls` tells the LDAP provider to use the STARTTLS command on port 389 to wrap communication in a secure layer.
At one point, we had "ldap_id_use_start_tls = True." This broke the AD lookup. After a lot of reading, we found a few articles that said to change it to False if you are using SSL certs. False, is the default setting.
To customize login information and restrict access, add FQDN in logon username and from home dir. Below are the defaults for the logn credentials.
- use_fully_qualified_names = True
- fallback_homedir = /home/%u@%d
The next several lines are parts of the config for the [domain/domain.local] section. I had them in the config but they caused trouble. I wanted to keep the commands for a record as I did a fair amount of research on them.
- # ldap_id_use_start_tls = True
- # dyndns_update = True
- # dyndns_refresh_interval = 3600
- # dyndns_update_ptr = True
- # dyndns_ttl = 3600
- # dyndns_auth = GSS-TSIG
Clear the contents of the existing config file.
Write the new contents of the existing config file.
Rocky - remove the cert path for Ubuntu.
Ubuntu - remove the cert path for Rocky.
Remove the comment for the cert path and verify.
Kerberos Config
All Distributions
Make a backup of the respective conf file.
The differeances between Rocky and Ubuntu were goofy. I opted to go through each setting and consolidate them into a single file.
Clear the contents of the existing config file.
Write the new contents of the existing config file.
Rocky - Remove the cert path for Ubuntu.
Ubuntu - Remove the cert path for Rocky.
Remove the includedir. Ubuntu will boot loop otherwise.
Remove the comment for the cert path and verify.
LDAPS Setup Client
Both distros use different LDAP paths for the config.
Rocky
Make a backup of the respective conf file.
Configure the LDAP config file:
Make sure the following items are not commented and/or configured. When the cert is installed into the system, it is added to a master list of certs. The cert order within the master list does not matter.
- BASE DC=domain,DC=local
- URI ldaps://domain.local:636
- TLS_REQCERT hard (default)
- TLS_CACERT /etc/pki/tls/certs/ca-bundle.crt
Add the cert locations:
Ubuntu
Make a backup of the respective conf file.
Configure the LDAP config file:
Make sure the following items are not commented and/or configured. When the cert is installed into the system, it is added to a master list of certs. The cert order within the master list does not matter.
- BASE DC=domain,DC=local
- URI ldaps://domain.local:636
- TLS_REQCERT hard (default)
- TLS_CACERT /etc/ssl/certs/ca-certificates.crt (default)
NSS Config
Fedora
At one time, I was using Fedora. This is a artifact left for historical purposes.
Fedora uses a different path than Rocky for the nsswitch.conf file.
I made changes to the equivalent of the Rocky user-nsswitch.conf file in Fedora, but the changes were not persistent after applying the policy. After testing, I do not beleive these changes are required for a client machine. They are more for file sharing permissions enforcement.
Rocky
Make a backup of the respective conf file.
In the nsswitch.conf, a comment states not to edit the nsswitch.conf because it takes lower presedance than /etc/authselect/user-nsswitch.conf. The comment also says to run an extra command to commit the changes. The command is in the "Final Touches" section.
Show the config file without comments.
Modify the config file so that each entity has "sss."
- passwd: sss files systemd (default)
- shadow: files sss (default)
- group: sss files systemd (default)
- services: files sss (default)
- netgroup: sss (default)
- automount: files sss (default)
- # initgroups: files (remove comment / add sss)
Set initgroups to files as a performance optimization to prevent group information from being fetched from Active Directory. You can omit that line. If you do, though, you may see delays when you list files or perform other actions that try to look up UID and GID information.
Show the config file without comments.
Ubuntu
Make a backup of the respective conf file.
Ubuntu does not have the silly message about modifying a different config file. So we can carry on our normal rounds.
Show the config file without comments.
Modify the config file so that each entity has "sss."
- passwd: files systemd sss (default)
- group: files systemd sss (default)
- shadow: files sss (default)
- services: db files sss (default)
- netgroup: nis sss (default)
- automount: sss (default)
- initgroups: files sss (added to config / add sss)
Set initgroups to files as a performance optimization to prevent group information from being fetched from Active Directory. You can omit that line. If you do, though, you may see delays when you list files or perform other actions that try to look up UID and GID information.
Show the config file without comments.
Configure NFS ID Mapping
Rocky
Make a backup of the respective conf file.
The ID Mapping dameon is used for translating ID numbers to usernames. This is something that gets installed by default with Rocky.
This file does not exist by default in Ubuntu until the nfs-kernel-server package is installed. So, do not worry that the file does not exist in the basic Ubuntu setup.
There is conflicting info on the [mapping] section. Some people use nobody vs nfsnobody. nfsnobody does not exist in either distro. The user "nobody" DOES exist in Ubuntu but a group "nobody" does not. So use "nogroup" instead. The nfsnobody user and the nfsnobody group map to the same ID that nobody and nobody/nogroup would.
Cat the file without comments:
Make the following sed changes.
Ubutnu
See the comments above for Rocky.
Home Directory Setup
Rocky
No changes are required for Rocky.
Ubuntu
realm does not enable pam_mkhomedir after the install. pam_mkhomedir creates home directories for network users after their intial login. To enable it, an extra command must be run which is documented in the "Final Touches" section.
After a new user logins in from AD, the default permissions allow for other users to read/write to otehr users home directories. Looking at multiple forums, we found several different methods for controlling this behavior.
Set the home directory permisssions for NEW logins:
Method 01
This method did NOT work.
Looking at the Rocky config, there is a option called "HOME_MODE" that we can add to the Ubuntu config. The following sed will accomplish this. The number sequeance at the end of the sed is a chmod and NOT a umask.
- sudo cat /etc/login.defs | grep -i HOME_MODE
- sudo cp /etc/login.defs /etc/login.defs.bak
- sudo sed -i '/^UMASK.*022/a\\n# HOME_MODE is used by useradd(8) and newusers(8) to set the mode for new\n# home directories.\n# If HOME_MODE is not set, the value of UMASK is used to create the mode.\nHOME_MODE 0700' /etc/login.defs
Method 02
This method did NOT work.
On another fourm, it was recommend to change a oddjob conf file. The following sed will accomplish this. The number at the end of the string is a UMASK number. It also must be a total of four digits.
- sudo cat /etc/oddjobd.conf.d/oddjobd-mkhomedir.conf | grep "oddjob/mkhomedir"
- sudo cp /etc/oddjobd.conf.d/oddjobd-mkhomedir.conf /etc/oddjobd.conf.d/oddjobd-mkhomedir.conf.bak
- sudo sed -i '/.*oddjob\/mkhomedir"/s/"$/ -u 0077"/' /etc/oddjobd.conf.d/oddjobd-mkhomedir.conf
Method 03
This method did NOT work.
The most popular responce on the forums was to modify dameon config file. We had a really hard time doing sed on this file until we figured out the file was using tabs all over the place. When applying the command in the "Final Section," there were warnings that system files were manually over-written and you would have to make all changes manually going forward unless you used --force to over-write the files back to defaults. It was gross. Method 04 works without any issues.
- sudo cat /etc/pam.d/common-session
- sudo cp /etc/pam.d/common-session /etc/pam.d/common-session.bak
- sudo sed -i '/session\trequired\tpam_unix.so .*/a session\trequired\tpam_mkhomedir.so skel=\/etc\/skel\/ umask=0077' /etc/pam.d/common-session
Method 04
This method DID work.
Create this file and run the command in "Final Touches" section." There will not be any issues.
Set the home directory permisssions for OLD logins:
localadmin's home directory was created during the intial OS install and is using default permissions. To make everything consistent, change the permission for the localadmin home directory.
Access Control
All Distributions
Restrict console access: Use the sssd.conf config to set restrictions based on Active Directory groups. Use the following setting: "ad_access_filter."
Restrict SSH access: Use the sssd.conf config to set restrictions based on Active Directory groups. Use the following setting: "ad_access_filter."
All of the important SSH server settings were set in the original template.
Even though you did not configure SSSD for authentication by including pam in the services list, end users may still be able to log in to the netboot server over SSH using PubkeyAuthentication or GSSAPIAuthentication methods. You may want to set an explicit limit for who can log in to your netboot server over SSH. DenyGroups users sudo echo "DenyGroups users" | sudo tee -a /etc/ssh/sshd_config
Restrict sudo access:
DO NOT edit /etc/sudoers.d/sudoers.
To restrict sudo access to a AD group, use one of the following methods:
Members of a group may gain root privileges: %ac-Linux-admins ALL=(ALL) ALL
Allows members of a group to execute any command: %ac-Linux-admins ALL=(ALL:ALL) ALL
Run the visudo command and append the following to the end of the file. Rocky seems to like tab formatting.
Final Touches
CentOS 7
Run the following command to set up NSS and PAM stacks. This will enable sssd and pam authentication on the required entries. CentOS 7's authconfig was replaced with authselect. The CentOS 7 command is below for posterity:
sudo authconfig --enablesssd --enablesssdauth --enablemkhomedir --update
Rocky
Display the current profile in use. The authselect select and apply-changes commands are self-explanatory. Verify that the changes have been applied to nsswitch.
Ubuntu
All Distros
Set permissions on the following files.
Set ownership on the following files.
Reboot the machine.
How To Login
Start SSH session as you normally would. For the username, do not use the domain name.
To verify who is logged in:
Leaving the domain
This will delete the computer object in AD and remove the local keytab file. This will also set the sssd.conf and krb5.conf files back to defaults.
NFS Server - NFS Install (Windows 2019)
If you want to install NFS on a Windows 2019 server, follow these instructions. That can be done to make CIFS shares also NFS shares.
A good use case for this would be to share a software or a ISO directory.
Install NFS server role:
- Open Server Manager.
- Click on Tools > Add Role and Features.
- Select Role-based or feature-based installation.
- Select the desigred server.
- Expand File and Storage Services > File and iSCSI Services.
- Select Server for NFS checkbox.
- Click next until you are done.
NFS Server Export (Windows 2019)
When you are managing NFS exports on a Windows 2019 server, you can use groups for machines. The following commands will create the group and the memebers so that the group can be used while setting up the permissions. AD is not used for this, rather local Powershell cmds.
To create a group of machines, open Powershell as admin and type:
AD Server - Configure NFS ID Mapping
The user ID and group ID must be set and be the same in the Linux AD joined machine and the Windows file server See "To verify AD users" section above for info on how to get the IDs.
- Open Active Directory and go to Users and Computers
- Go to: View > Check Advanced Features
Go to the User you need to add the ID to. Go to "Attribute Editor" tab and make sure the Filter: "Show only attributes that have values" and the "Show only writeable attributes" are not checked. Scroll through the list and change the following variables so they are the same between the Linux machine's id and AD:
- gidNumber
- uidNumber
Alertnaitviely, you can use the following commands to add the IDs to AD. The only catch is that the commands must be run in Powershell on a Windows server that is joined to the domain and that has the NFS server role installed. So editing the objects in AD may be easier.
Use the following Powershell command to verify the user IDs. This must be run from a Windows NFS server that is joined to the same AD domain.
Use the following Powershell command to verify the group IDs. This must be run from a Windows NFS server that is joined to the same AD domain.
You can also add the default shell and home directory values using these attributes. Some articles said they are required.
NFS client setup (Windows 2019)
If you want to mount the Linux NFS exports on a Windows 2019 server, the NFS Server must be installed. Type the following command in powershell. This will map the IDs you entered earlier in AD to the Linux IDs.
NFS client setup (Windows 10)
Go to the Classic Control Panel > Programs and Features Turn Windows features on or off. Check Services for BOTH NFS Administrative Tools AND Client for NFS must be checked a reboot may be required.
Alternatively, you can install Windows 10 NFS Client using Powershell in admin mode.
The following is what will force the ID mappings to use Active Directory. Open the Windows 10 app "Services for Network File System." Right click on "Services for NFS." Check the box for: "Active Directory domain name:" and type: domain.local
This can also be achieved by Powershell in admin mode:
To show a list of current settings, open a Powershell prompt in admin mode:
OPTIONAL: Set the UID and GID.
The username/group cannot be set during the mounting of an export. You have to use Powershell in admin mode. None of the docs referenced a HKCU key, only HKLM. A reboot is required for these changes to take effect.
Some documentation said that to add "-o anon nolock" to the Windows mount command. During testing, this did not seem to be required. Example:
NFS mount commands (Windows)
To get a list of NFS shares, open a Windows cmd prompt:
To mount a NFS share, open a Windows cmd prompt:
To get a list of current mounts and properties, open a Windows cmd prompt:
To unmount a NFS share, open a Windows cmd prompt:
NFS client setup (Linux)
Install the NFS client in the respective distro.
Rocky
Ubuntu
NFS mount commands (Linux)
To get a list of NFS shares:
To mount a NFS share, create the folder, and then mount the share to the folder:
You will likely have to play with permissions, referance the chmod, chown, and chgrp commands above.
To unmount a NFS share:
Troubleshooting
To increase verbosity of errors, up the logging level. The debug level can be set for the domain, NSS, or PAM. The sed command will enforce this for all three. Don't forget to change it back to the default of 2.
Logs to tail for troubleshooting:
To get a list of the current kerberos tickets:
If the verify is failing, use the following commands to flush sss_cache or delete the files (to clear the SSSD cache).
From the Ubuntu docs, you can clear the cache by using the sssctl tool. Remove the whole cache:
One element:
Expire everything:
References
How SSSD works
- https://www.ateam-oracle.com/part-1-of-4-sssd-linux-authentication-introduction-and-architecture
- https://www.ateam-oracle.com/part-2-of-4-sssd-linux-authentication-ldap-identity-store-requirements
- https://www.ateam-oracle.com/part-3-of-4-sssd-linux-authentication-implementation-step-by-step-guideline
- https://www.ateam-oracle.com/part-4-of-4-sssd-authentication-known-problems-and-troubleshooting-tips
Basic AD join setup
- https://geekdudes.wordpress.com/2019/04/26/centos-7-windows-active-directory-integration-using-sssd/
- https://www.rootusers.com/how-to-join-centos-linux-to-an-active-directory-domain/
- https://www.redhat.com/sysadmin/linux-active-directory
- https://yallalabs.com/linux/how-to-join-centos-7-rhel-7-servers-to-active-directory-domain/
- https://www.ateam-oracle.com/using-sssd-with-kerberos-and-active-directory-to-terminal-into-an-oci-linux-machine
- https://ubuntu.com/server/docs/service-sssd
How to install CA certs
- https://www.redhat.com/sysadmin/ca-certificates-cli
- https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/security_hardening/using-shared-system-certificates_security-hardening
- https://gist.github.com/kekru/deabd57f0605ed95d5c8246d18483687
- https://askubuntu.com/questions/645818/how-to-install-certificates-for-command-line
- https://www.techrepublic.com/article/how-to-install-ca-certificates-in-ubuntu-server/
AD join with SSL setup
- https://geekdudes.wordpress.com/2020/02/18/linux-connecting-to-windows-ldap-over-ssl-ldaps-using-certificate/
- https://yourlinuxguy.com/?p=635
- https://support.google.com/cloudidentity/answer/9089736?hl=en#zippy=%2Copenldap-ldapsearch-linux%2Csssd-red-hat-enterprise-and-centos
- https://kifarunix.com/configure-sssd-for-ldap-authentication-on-ubuntu-20-04/
AD GPO light reading
- https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/windows_integration_guide/sssd-gpo
- https://sssd.io/docs/design_pages/active_directory_gpo_integration.html
- https://www.systutorials.com/docs/linux/man/5-sssd-ad/
Securing files, configuring NSS and PAM
- https://www.thegeekdiary.com/how-to-connect-to-an-active-directory-domain-using-realmd-configure-centos-rhel-7-as-active-directory-client/
- https://www.golinuxcloud.com/add-linux-to-windows-ad-domain-adcli-centos-7/
- https://www.2daygeek.com/join-integrate-rhel-centos-linux-system-to-windows-active-directory-ad-domain/
Realm customization
SSSD config
- https://access.redhat.com/solutions/262923
- https://access.redhat.com/solutions/4775881
- https://www.redhat.com/sysadmin/linux-active-directory
- https://sssd.io/docs/design_pages/active_directory_access_control.html
ldap.conf Config
How to flush and clear SSSD cache
Customize SSH access
- https://www.rootusers.com/configure-additional-options-described-in-documentation-for-ssh-in-linux/
- https://patrikwm.github.io/2016/11/11/001-Centos-SSH-Active-Directory.html
ID Mapping
- https://fedoramagazine.org/secure-nfs-home-directories-kerberos/
- https://tbellembois.github.io/kerberos.html
- https://access.redhat.com/solutions/2677831
- https://cabinet496.rssing.com/chan-23501210/all_p1.html
- https://docs.microsoft.com/en-us/powershell/module/nfs/new-nfsmappedidentity?view=windowsserver2019-ps&viewFallbackFrom=win10-ps
Windows - Create NFS client groups
Access to File System using UNC Path is Slow or Fails
Fantastic NFS sites
- https://cabinet496.rssing.com/chan-23501210/all_p1.html
- https://blog.edie.io/2018/06/16/mounting-nfs-shares-in-windows-using-identity-mapping/
- http://thewindowsupdate.com/2019/04/10/server-for-nfs-diagnostics/
- https://techcommunity.microsoft.com/t5/storage-at-microsoft/nfs-identity-mapping-in-windows-server-2012/ba-p/424602
NFS Client for Windows 10
- https://stealthbits.com/blog/mounting-nfs-exports-unix-server/
- https://docs.microsoft.com/en-us/powershell/module/nfs/set-nfsmappingstore?view=windowsserver2019-ps
"KDC has no support for encryption type" troubles
- https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/installing_identity_management/installing-trust-between-idm-and-ad_installing-identity-management
- https://ldapwiki.com/wiki/MsDS-SupportedEncryptionTypes
- https://www.stigviewer.com/stig/windows_server_2019/2019-12-12/finding/V-93495