TL;DR: Don't do this. A colleague recommended me not doing this, but dump users data and recreate users in FreeIPA which is much simpler, quicker and more reliable. And now I agree with him. For I have aged.
If you do, however, plan to do this for whatever reasons, lots of text under the cut
Preamble
In my spare time I dabble in server management as a hobby to upkeep some services for me and others. Started it while being a student, now it's just dragging along since not many takers for unpaid sysadmin work where everyone complains and hates you when something doesn't work, but no one remembers you when nothing keeps on failing. Or everything keeps on not failing.
This particular setup involved Atlassian Crowd as a login source for other Atlassian services as well as some on- and off-site Apache hosts and java services. As an extra point of pain for HR people, accounts had to be made in at least 3 different places - Crowd, Google, a Windows server which hosted SVN repos and Win-only executables; and a slew of Linux machines on need-to-have basis. I've been looking at this mess for years and begging to get an extra machine to set up some kind of LDAP solution. I finally got a rig with FreeIPA on it and an LDAP tree. Great. Priority of this thing was low and I was kinda-sorta avoiding it, because it seemed large and tedious work with unfamiliar tech. Now a wild motivator appeared in the form of Windos machine being kicked out of the server room, so I was forced to migrate SVN. Migrating SVN to a Linux machine with somewhat dynamic user base means that I really don't want to have to edit authz files every time a new member joins. Or create users on that particular machine. So, an LDAP has an actual pressing need for it to be set up! Yay!
Why I didn't want to just create new users in FreeIPA? I didn't know that there are no actual UUIDs exposed from Crowd. Users are identified by their UID field, maaaaybe external UUID used for username change tracking if correctly set up in connector. I thought that we'll lose all the user content (tasks, pages, wikis) mapping to their users and creators. Which is not true. So, once again - avoid this.
Step 1. Know your enemy.
Since I had no clue what I'm dealing with, a clue was needed. I did read first few chapters of IBM Red Book that deal with generic stuff and felt confident that we can do this. Looked at FreeIPA UI, clicked about, basics seemed simple enough, and the whole UI looks like CRUD shell around LDAP.
Crowd is a rather familiar beast already, since I've been dragging it places by the tail for years (endless upgrades, migration from windos to Linux, migration from MySQL to Postgres, patches and DB hotfixes). Just in case did some searching around for discussions, saw that there's this entry about generic LDAP and then there's this entry about DB hacks and then there's this request for FreeIPA support and then there's this guy and then there's this entry that hints that what you want to do is DOOOOOOMED! They do say that you can beg for source code if you buy license, but I was looking at a server with user data that's gonna fly out of the server room window in a week. I don't have time to remember how Java works and then dig through Atlassian's codebase, even if they provide it right away.
Step 2. Familiarize yourself with the playing field.
I didn't get FreeIPA root, but I got a functioning user with most permissive role (I think). Never actually looked into those levels, it turned out to be enough). IPA instance had anonymous searches blocked, but whatevz, we have a almost-root user! A stump-user! We should be able to search and find at least ourselves using ldapsearch from ldap-utils package:
$ ldapsearch -ZZ -h im.shirtec.com -D "uid=kaspars,cn=users,cn=accounts,dc=shirtec,\
dc=com" -b "dc=shirtec,dc=com" -W -x '(&(objectclass=inetorgperson)(uid=kaspars))' Enter LDAP Password: # extended LDIF # # LDAPv3 # base <dc=shirtec,dc=com> with scope subtree # filter: (&(objectclass=inetorgperson)(uid=kaspars)) # requesting: ALL # # kaspars, users, accounts, shirtec.com dn: uid=kaspars,cn=users,cn=accounts,dc=shirtec,dc=com givenName: Kaspars sn: Laizans uid: kaspars cn: Kaspars Laizans displayName: Kaspars Laizans initials: KL gecos: Kaspars Laizans gidNumber: 1568000017 objectClass: top objectClass: person objectClass: organizationalperson objectClass: inetorgperson objectClass: inetuser objectClass: posixaccount objectClass: krbprincipalaux objectClass: krbticketpolicyaux objectClass: ipaobject objectClass: ipasshuser objectClass: ipaSshGroupOfPubKeys objectClass: mepOriginEntry loginShell: /bin/bash homeDirectory: /home/ipa_test2 ... blah ... uidNumber: 1568000020 mail: xxx@yyy.zz title: Big Boss krbLoginFailedCount: 0 krbLastFailedAuth: 20220114182321Z # search result search: 3 result: 0 Success # numResponses: 2 # numEntries: 1
Whoop, whoop! Only problem is that it took like 10 minutes to get response. The aforementioned colleague who set up the machine said that it works fine. From his machine, from the server itself, but not for me. From anywhere but the server itself. Turns out there's FreeIPA self-issued CA cert that you need importing:
$ wget --no-check-certificate -O im_shirtec_com_CA.crt https://im.shirtec.com/ipa/config/ca.crt $ sudo cp im_shirtec_com_CA.crt /etc/ssl/certs $ sudo cp im_shirtec_com_CA.crt /usr/share/ca-certificates # Add this cert to /etc/ca-certificates.conf $ sudo update-ca-certificates --verbose
Once you're done with this, responses become instantaneous. Good, good, feels like progress.
3. To war, my trusty steed!
Now Crowd! Franz Geffke wrote, that he was successful in connecting Crowd to IPA. He forgot to mention some tiny details. Or rather forgot to mention that his setup is very specific. The fact, that he uses only LDAP, not IPA bit of FreeIPA. At least with that setup.
Ok, let's reproduce his work then.
Create connector. Connector allows to get group memberships from LDAP, while Delegated just gets auth info, but groups are stored locally. My parameters and understanding about them:
Name/description/caching is up to you.
Generic Directory server. I guess for others there are some specific tweaks/nomenclature. Since Franz's tutorial used Generic, I went with that as well.
Connector URL: ldap://im.shirtec.com:389/ - your LDAP server
SSL - I have it behind VPN + certs havent been arranged yet, for now None
Use nested groups - later found out that this works best, if you have nested groups within IPA (my conflu-users/jira-users are group-members of ipausers group to allow access to Atlassian services to all users by default).
Encryption PLAINTEXT - yes, i know it's the worst in combination with no SSL, but to get something going good enough behind VPN. Later will add security layers.
Base DN: dc=shirtec,dc=com - root of the tree where to look for objects
Username: uid=crowd_reader,cn=users,cn=accounts,dc=shirtec,dc=com - user that has read access to the tree, create specific in FreeIPA
Password: crowd_reader user password that is used for authenticating against LDAP to get access to the tree
This is where our approaches start to differ, I got here with a bit of trial and error:
User DN: cn=accounts. Franz's suggestion should work, but data is fetched for both users and groups, then filtered to provide only accounts with objectClass=inetorgperson. I use here cn=users,cn=accounts, since I know that from the ldapsearch query before. And it's what FreeIPA uses. If you have these wrong, then IPA might not see your users. And this is the bit of info Franz omits, that he does not use FreeIPA UI to manage those users. I really don't believe that he has them in IPA, because I tried.
User name attribute: uid
User name RDN attribute: cn
These 2 are a bit confusing. When you get Crowd running against LDAP with these settings and create users in Crowd, you can create duplicate entries in FreeIPA like:
dn: cn=ipa_test6,cn=users,cn=accounts,dc=shirtec,dc=com for crowd dn: uid=ipa_test6,cn=users,cn=accounts,dc=shirtec,dc=com for IPA
Both are valid and both are different. Both then appear in Crowd as 2 different users ipa_test6 and ipa_test6#1
On the other hand if you swap them and make
User name attribute: cn
User name RDN attribute: uid
wil create correct IPA-style entry:
dn: uid=ipa_test6,cn=users,cn=accounts,dc=shirtec,dc=com
Then if you try to make another user with the same UID in IPA, you'll get complaint that user exists. Which it does. But FreeIPA does not see it.
User unique identifier attribute: ipaUniqueID is used by FreeIPA. Not tested tho.
Group DN: cn=groups,cn=accounts - same as for users
Group object class: ipausergroup
Group object filter: (objectclass=ipausergroup)
Group name attribute: cn - this follows FreeIPA nomenclature and works fine, unlike with user name attribute
Group members attribute: member
Save, import from internal, activate, and you're good. Just as Franz said.
EXCEPT
You can't import internal directory users, because their password hashing methods are different. Atlassian uses SHA1-512 with custom salt, that is not supported by OpenLDAP running behind IPA. Well, shucks. Luckily, there's a hack provided by Atlassian to enable us to copy over users to LDAP. Nice. And it even works!
EXCEPT
It fills LDAP with minimalistic user info. Basically, name, email, uid, password. On top of that, latter is useless. What you get in LDAP is:
# ipa_test8, users, accounts, shirtec.com dn: uid=ipa_test8,cn=users,cn=accounts,dc=shirtec,dc=com mail: ipa_test8@xxx.yyy uid: ipa_test8 displayName: Ipa Test8 givenName: Ipa objectClass: inetorgperson objectClass: organizationalPerson objectClass: person objectClass: top objectClass: nsMemberOf sn: Test8 cn: ipa_test8 memberOf: cn=jira-users,cn=groups,cn=accounts,dc=shirtec,dc=com
If you compare to my IPA user above, you'll notice a bunch of objectClasses and their attributes missing. As much as I could figure out, ipaobject and posixUser classes and their attributes are mandatory for IPA.
Ok, but we got all our 100s of users into LDAP, it's already progress, right? RIGHT? There's ldapmodify after all. If we could modify entries, then we could write a script that generates ldif files to execute on this. Except I didn't have LDAP root access to be able to do it.
Now what? I have half-baked users in LDAP that I can read, but not modify in any way known to me. Turns out, theres migrate-ds tool for IPA. That, according to documentation, can read in external LDAP source, reformat it and import into FreeIPA. We are saved! (Quite literally).
$ ipa migrate-ds ldaps://im.shirtec.com --bind-dn\="uid=kaspars,cn=users,cn=accounts,dc=shirtec,dc=com" --with-compat \--user-container="cn=users,cn=accounts,dc=shirtec,dc=com" --group-container\="cn=groups,cn=accounts,dc=shirtec,dc=com"
EXCEPT
To import using migrate-ds script, you need to have objectClass=posixAccount on your entries. With UUID and GUID mandatory. That's what script complains about. Ok, let's generate those programmatically, bypassing POSIX user error reports:$ sudo vi /usr/lib/python3.6/site-packages/ipaserver/plugins/migration.py if 'gidnumber' not in entry_attrs: #raise errors.NotFound(reason=_('%(user)s is not a POSIX user') % dict(user=pkey)) entry_attrs['gidnumber'] = '1568000005' import random entry_attrs['uidNumber'] = ('%d' %(1568000003 + random.randint(1,1000000))) $ sudo systemctl restart httpd
There I took existing group (ipausers) guidnumber and uidNumber range from FreeIPA server config, added last uidNumber I could find (3) and then add a random number to it. Depending on number of users you're trying to import, you might get duplicates, but those should be easy to find later. YMMV.
Now it complains about existing users. As expected, you can't load data into FreeIPA from its own LDAP, because that's where it dumps modified users with the same UIDs.
Ok, we need an intermediate LDAP where to dump users from Crowd before loading into FreeIPA. Set up separate bare-minimum OpenLDAP instance, add connector to it in Crowd, duplicate internal directory into it, sync. In Crowd logs I see "syncing.... deleted 587 users". THATS ALL OF THEM! Luckily that was a duplicate that got emptied out.
I guessed, that target LDAP was newer than the source directory and Crowd pulled all the changes. Changes being - source of changes is empty. OK.
Add connector to intermediate LDAP in Crowd, duplicate internal directory into it, don't sync. Now we have internal data store of "LDAP format" (I know it's not) data hanging in Crowd. Create another connector to intermediate LDAP, import users from first intermediate connector into second, now timestamps are newer on Crowd end. Sync second and VOILA!
Now our hacked migrate-ds with extra --continue param manages to import. Well, most of the stuff. It looks like it missed some user-group mappings, but that's minor. Now they all are in our FreeIPA.
EXCEPT their passwords. Because they are in the wrong format. And users are missing Kerberos tickets. So we are almost back to square 1 - export usernames/emails from Crowd DB into CSV and write a script that generates temporary passwords, changes user passwords in FreeIPA using ipa passwd and sends them away to users via email with instructions to go to FreeIPA migration page. Not that much less (or more) work than writing script for creating users and sending passwords to them in the first place.
Another feature you should be aware of, if you use nested groups - group nesting has to be enabled not only on Crowd connector, but also in EACH of the Atlassian applications using this Crowd instance. Otherwise you get following sequence:
- Crowd gets users/groups/nested groups from LDAP
- When admin is looking at user group membership in application (Jira/conflu/whatever), it shows group info from Crowd (including nested) as Crowd sees it.
- When user logs in, user groups are fetched from LDAP via crowd (if "Every time user logs in" is selected, in stead of "For newly added users only". If Application doesn't have "use nested groups" checked, it ignores them and updates internal cache with only parent group
- Then looking at user group membership in application you see only parent group
Ohh, this was 3 or 4 long evenings I'm never gonna get back. But at least now I'm familiarized with LDAP and more intimately with Crowd. For when I need it next time in like 20 years maybe.
No comments:
Post a Comment