Tuesday, November 30, 2010

Interoperability between Windows and OpenIndiana shares using ZFS & ACL (and Solaris 11)

What I have always wanted to do is to have one location for all my users to store there files.
At the moment we have a Windows Home filesystem and Solaris Home filesystem and even though samba does a good job I have never been very happy with it. This is where ZFS and the ACL which is part of the NFS v4 allow me to finally achieve Windows & Solaris Interoperability.

This has been tested on OpenIndiana oi_147 and Windows Server 2008 running "Active Directory"

In this example:
Domain              = test.int
Windows 2008 server = windows     = 192.168.56.3
Solaris Server      = openindiana = 192.168.56.5
Setup Windows Server
-- Windows 2003 would do
-- Windows 2008 with patches
-- NTLMv2 authentication problem:
      http://support.microsoft.com/kb/957441/
-- Windows Server 2008 SP1 with Microsoft Kerberos hot fix KB951191:
-- or Windows Server 2008 SP2


Setup Solaris Server
-- Check DNS setup
# cat  /etc/resolv.conf
domain  test.int
nameserver  192.168.56.3
# grep  dns  /etc/nsswitch.conf
hosts:      files dns mdns
ipnodes:    files dns mdns 
-- Check software installed (SMB Server libraries and commands)
% pkg list smb
NAME (PUBLISHER)                              VERSION         STATE      UFOXI
service/file-system/smb                       0.5.11-0.148    installed  -----
system/file-system/smb                        0.5.11-0.148    installed  -----
If not then install it
$ pfexec install service/file-system/smb
-- Sync clocks on network (any time difference 5mins can cause problems)
$ pfexec ntpdate DC-host
or
$ pfexec rdate time-host
in our case:
$ pfexec ntpdate windows
--  Set up mapping of users and groups between systems
    Showing the most basic setup:

$ svcs \*idmap\*
STATE          STIME    FMRI
disabled       12:16:59 svc:/system/idmap:default
$ svcadm enable idmap
$ svcs \*idmap\*
STATE          STIME    FMRI
online         12:40:38 svc:/system/idmap:default
$ pfexec idmap add 'winuser:*@test.int' 'unixuser:*'
$ pfexec idmap add 'wingroup:*@test.int' 'unixgroup:*'
$ idmap list
add     winuser:*@test.int    unixuser:*
add     wingroup:*@test.int    unixgroup:*

-- Configure Kerberos: (AD uses Kerberos authentication)
    Edit /etc/krb5/krb5.conf and specify the fully qualifed AD domain name, in UPPERCASE, as the default realm. Also, specify the fully qualified host name for the domain controller at the kdc, admin_server and kpasswd_server

edit  /etc/krb5/krb5.conf to look like:
    [libdefaults]
        default_realm = TEST.INT

    [realms]
        TEST.INT = {
                kdc = windows.test.int
                admin_server = windows.test.int
                kpasswd_server = windows.test.int
                kpasswd_protocol = SET_CHANGE
        }

     [domain_realm]
         .test.int = TEST.INT

-- Start the smb (CIFS) services and check it is running
$ pfexec svcadm enable -r smb/server
$ svcs \*smb\*
STATE          STIME    FMRI
disabled       Nov_08   svc:/network/smb/client:default
online         15:21:01 svc:/network/smb/server:default
online         15:21:03 svc:/network/shares/group:smb
-- Need to join the AD domain with a Domain Administrator access
    If you are using Windows 2008 Domain then you will need an extra step.

$ ping windows
windows is alive 
$ pfexec smbadm join -u Administrator test.int
After joining test.int the smb service will be restarted automatically.
Would you like to continue? [no]: yes
Enter domain password:
Joining test.int ... this may take a minute ...
failed to find any domain controllers for test.int

$ tail /var/adm/messages
...openindiana smbd[1101]: [ID 700049 daemon.error] smbd: failed locating domain controller for test.int
....openindiana smbd[1134]: [ID 702911 daemon.notice] smbd_dc_update: test.int: located windows
....openindiana smbd[1134]: [ID 702911 daemon.notice] Failed to establish NETLOGON credential chain


Set the LAN manager authentication level on your Solaris system

$ pfexec sharectl set -p lmauth_level=2 smb
$ pfexec smbadm join -u Administrator test.int
After joining test.int the smb service will be restarted automatically.
Would you like to continue? [no]: yes
Enter domain password:
Joining test.int ... this may take a minute ...
Successfully joined test.int

Setup Solaris filesystem
As we said we are after a single fire-system (share) which works on both Operating System, so that any permissions work on both platforms and are transferable to the other one:

  -- Enable Cross-Protocol Locking
      - SMB assumes mandatory locking
      - UNIX advisory locking
  -- Mixed case
  -- Enable SMB sharing on share
$ pfexec zfs  create  -o nbmand=on  -o  casesensitivity=mixed rpool/export/homes

repeat next 2 lines cd for all users
$ pfexec zfs create rpool/export/homes/andrew
$ pfexec zfs set  sharesmb=name=andrew  rpool/export/homes/andrew

$ sharemgr show -vp
default nfs=()
smb smb=()
    * /var/smb/cvol     smb=()    ""
          c$=/var/smb/cvol     smb=(abe="false" guestok="false")    "Default Share"
zfs
    zfs/rpool/export/homes/andrew smb=()
      andrew=/export/homes/andrew

$ zfs get  nbmand,casesensitivity,sharesmb  rpool/export/homes/andrew
NAME                      PROPERTY         VALUE        SOURCE
rpool/export/homes/andrew  nbmand           on           inherited from rpool/export/homes
rpool/export/homes/andrew  casesensitivity  mixed        -
rpool/export/homes/andrew  sharesmb         name=andrew  local

$ pfexec chown  andrew:staff  /export/homes/andrew
 -- watch out for which ls / chmod commands
$ touch file1
$ which ls
/usr/gnu/bin/ls
$ ls -la
total 10
drwxr-xr-x   3 andrew   staff          4 Nov 18 18:42 .
drwxr-xr-x   4 root     root           4 Nov 18 18:33 ..
drwxr-x---   2 root     sys            3 Nov 18 18:34 .$EXTEND
-rw-r--r--   1 andrew   staff          0 Nov 18 18:42 file
$ /usr/bin/ls -lv
total 1
-rw-r--r--   1 andrew   staff          0 Nov 18 18:42 file
     0:owner@:read_data/write_data/append_data/read_xattr/write_xattr
         /read_attributes/write_attributes/read_acl/write_acl/write_owner
         /synchronize:allow
     1:group@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow
     2:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize
         :allow
$ /usr/bin/ls -lV
total 1
-rw-r--r--   1 andrew   staff          0 Nov 18 18:42 file
                 owner@:rw-p--aARWcCos:-------:allow
                 group@:r-----a-R-c--s:-------:allow
              everyone@:r-----a-R-c--s:-------:allow


-- Identity Mapping...
    - Unknown Windows identities are mapped to dynamically alloctaed UIDs/GIDs 
    - Unnown Unix identities are not mapped to Windows so they MUST exist in AD.

i.e.
idmap[501]: [ID 523480 daemon.notice] AD lookup of winname root@test.int failed, error code -9961
idmap[501]: [ID 523480 daemon.notice] AD lookup of winname sys@test.int failed, error code -9961
idmap[501]: [ID 523480 daemon.notice] AD lookup of winname staff@test.int failed, error code -9961

    - It is a good idea that well know accounts which may be used in ACL have a permanent map to a UNIX group
    i.e. lets bind UNIX group "winadmin"to Windows Group
$ idmap add "wingroup:Domain Admins@test.int"  unixgroup:winadmin

-- ACL Property on filesystem
   discard - New objects, no ACL entries are inherited
   noallow - New objects, only inheritable ACL entries that have access to type deny are inherited.
    restricted - New objects, the write_owner and write_acl permissions are removed when ACL entry is inherited.
    passthrough - New objects are created with a mode determined by the inheritable ACEs (Access Control Entries). [Sorry what is that in english]
    passthrough-x - As above, plus files are created with the execute (x) set.
$ pfexec zfs get aclinherit rpool/export/homes
NAME                        PROPERTY    VALUE          SOURCE
rpool/export/homes          aclinherit  restricted     default

$ pfexec zfs set aclinherit=passthrough-x rpool/export/homes
$ zfs get aclinherit  rpool/export/homes/andrew
NAME                      PROPERTY    VALUE          SOURCE
rpool/export/homes/andrew  aclinherit  passthrough-x  inherited from rpool/export/homes


ACL TABLE to be inserted here

What has changed in recent versions
    - deny ACL are not required in most cases now. exceptions: 0705 (g-rwx), 0060 (u-rwx)
    - aclmode has gone, which means that chmod will discard all ACLs
    - user and owner are treated together?


Possible Solution
    - Owner needs the correct permissions
    - Group needs the correct permissions
    - User Andrew

$ /bin/ls -ldv /export/homes/andrew
drwxr-xr-x   3 andrew   staff          3 Nov 30 12:40 /export/homes/andrew
     0:owner@:list_directory/read_data/add_file/write_data/add_subdirectory
         /append_data/read_xattr/write_xattr/execute/read_attributes
         /write_attributes/read_acl/write_acl/write_owner/synchronize:allow
     1:group@:list_directory/read_data/read_xattr/execute/read_attributes
         /read_acl/synchronize:allow
     2:everyone@:list_directory/read_data/read_xattr/execute/read_attributes
         /read_acl/synchronize:allow

$ /bin/ls -ldV /export/homes/andrew
drwxr-xr-x   3 andrew   staff          3 Nov 30 12:40 /export/homes/andrew
                 owner@:rwxp--aARWcCos:-------:allow
                 group@:r-x---a-R-c--s:-------:allow
              everyone@:r-x---a-R-c--s:-------:allow

-- So, A0 refers to position 0, A1 refers to position 1, etc....
-- The following commands will replace the ACL at position 0,1,2 with new ones
-- The only change is to add inheritance to the directories

pfexec /bin/chmod "A0=owner@:rwxp--aARWcCos:fd:allow" /export/homes/andrew
pfexec /bin/chmod "A1=group@:r-x---a-R-c--s:fd:allow"  /export/homes/andrew
pfexec /bin/chmod "A2=everyone@:r-x---a-R-c--s:fd-----:allow"  /export/homes/andrew

-- Make life easier lets do it as user andrew and gnu removed from path:
-- Add user andrew (owner) to the ACL
-- Add group staff (group) to the ACL

$ chmod "A+user:andrew:rwxpdDaARWcCos:fd-----:allow" /export/homes/andrew
$ chmod "A+group:staff:r-x---a-R-c--s:fd:allow"  /export/homes/andrew
$ chmod "A+group:winadmin:rwxpdDaARWcCos:fd-----:allow"   /export/homes/andrew

$ ls -ldV /export/homes/andrew
drwxr-xr-x+  3 andrew   staff          3 Nov 30 12:40 /export/homes/andrew
         group:winadmin:rwxpdDaARWcCos:fd-----:allow
            group:staff:r-x---a-R-c--s:fd-----:allow
            user:andrew:rwxpdDaARWcCos:fd-----:allow
                 owner@:rwxp--aARWcCos:fd-----:allow
                 group@:r-x---a-R-c--s:fd-----:allow
              everyone@:r-x---a-R-c--s:fd-----:allow

Hope it works....

Ref:
- Jarod Nash (LOSUG slides)
- http://wiki.genunix.org/wiki/index.php/CIFS_Service_Troubleshooting
- http://arc.opensolaris.org/caselog/PSARC/2010/029/20100126_mark.shellenbaum

5 comments:

davekoelmeyer said...

Great to see practical articles using OpenIndiana - keep it coming!

Anonymous said...

Hi.
I've tried with a 2008sp2 and a sxce11, without success...
Also tried to change (downgrade) the "sharectl" level, without success...
Have You any other suggestion?

Still have message:
pfexec smbadm join -u Administrator domain.local
After joining domain.local the smb service will be restarted automatically.
Would you like to continue? [no]: yes
Enter domain password:
Joining domain.local ... this may take a minute ...
failed to find any domain controllers for domain.local

And in the messages:
Jan 31 18:06:21 sxce_11 smbd[716]: [ID 663266 daemon.notice] DNS query for _ldap._tcp.dc._msdcs failed: Host name lookup failure
Jan 31 18:06:21 sxce_11 smbd[716]: [ID 702911 daemon.error] cannot locate domain controller for domain.local

But, from klist:
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: Administrator@DOMAIN.LOCAL

Valid starting Expires Service principal
28/01/11 21.13.08 29/01/11 07.13.14 krbtgt/DOMAIN.LOCAL@DOMAIN.LOCAL
renew until 04/02/11 21.13.08
28/01/11 22.03.52 29/01/11 07.13.14 ldap/domainad.domain.local@DOMAIN.LOCAL
renew until 04/02/11 21.13.08

So what's the matter?
Thanks!

Andrew Watkins said...

Couple of thoughts.
- Time correct on both hosts
- Check DNS from both systems:
-- nslookup WINDOWS_SERVER
-- nslookup Solaris_SERVER
-- ping WINDOWS_SERVER
-- ping SOLARIS_SERVER

I will have think.

Anonymous said...

well... I don't know what's the modify which has solved the issue.

All changes I've made still now:
- AllowLegacySrvCall new reg key in system windows
- new wins services in windows
- regenerate the dns in windows
- modify of "search" straing into the "resolv.conf"
- may be something I don't remember for now...

But now it works!
:)
Thanks!!!!

(of course I'm going to redo all changes one for time, untill I'll understand....)

n4th4nr1ch said...

pfexec sharectl set -p lmauth_level=2 smb

There is no longer an lmauth_level. More info here: http://www.room425.com/?m=201205

Basically:
sharectl set -p client_lmauth_level=2 smb
sharectl set -p server_lmauth_level=2 smb