I have been running Samba on Solaris 11 zone for a while now and I thought I would finally write the instructions up. First, why am I using samba and not the CIFS server which comes with Solaris now. Well, one reason.... CIFS does not work in a zone and Samba does. I have our NFS file-server in a zone so I needs SMB access to those files.
Before we can start you need change the "Maximum number of supplementary groups (groups that a user can be in, additionally to the primary group)".
I have just spotted that this has changed or at least documented in Solaris 11.1, but is still required in /etc/system ref: http://docs.oracle.com/cd/E26502_01/html/E29022/chapter2-4.html#gjmtw & http://www.c0t0d0s0.org/archives/7634-Less-known-Solaris-11.1-features-A-user-in-1024-groups-and-a-workaround-for-a-25-year-old-problem.html
# echo "set ngroups_max=512" >> /etc/system
Reboot the server.