RTC2GIT – Rational Team Concert to GIT migration (part 1)

Introduction

Rational Team Concert (RTC) is a great, extensive and rather complicated configuration management (CM) system. Now after several years, it is time for us to move on to GIT. Git is simpler to use, more straight forward in its approach and cheaper. We have many project areas in RTC, each with many streams with many components, each stream with a long history of changes, baselines and snapshots, some of them over 5 years of history. To switch to Git, we want to migrate the history from RTC to GIT and we need a tool for that. In this article (serie) I will explain how we did it.

On the internet I found 2 solutions:

  1. rtc2git tool based on the RTC CLI (Command Line Interface), implemented in Python. See Github. There is also a version in Java.
  2. rtc2git tool based on the RTC REST API (Application Programming Interface), implemented in Bash scripts. See BitBucket

The CLI-based tool is buggy because the RTC CLI is buggy. This is a known issue at IBM. Istvan Verhas implemented an API-based tool in bash, see this article. I could not find his bash scripts, but in general I find bash scripts to be rather cryptic and unreadable, especially if it contains a lot of awk and sed commands. So, using an example of an RTC changeset exporter in Python, I decided to build my own RTC2GIT tool in Python.

At first I was a bit reluctant to dive into Python since I had not prior experience with Python. I do have some experience with Perl (over 20 years ago). To be honest, Python turns out to be a lot easier to learn than Perl, as long as you set your editor to replace tabs by spaces – Python is quite critical concerning indentations. Mixing tabs and spaces results in errors which are hardly visible for the human eye.

The real challenge was that requirements for the RTC2GIT tool evolved over time. Starting with a simple migration of a single component from a single stream, it quickly turned into a complex of requirements, e.g. multiple components, multiple streams, loadrules, incremental migration (continued development in RTC after migration to the history to Git) and adding branches (RTC streams branched off from another stream to branches in Git). It turned out to be too much for a single article, so I decided to write it down in separate articles. This is the first, introductory article where you will get a general overview.

General concepts

As I wrote in the introduction, I go for an RTC2GIT tool using the RTC REST API and written in Python. Basically, it consists of 2 tools:

  1. RTC_export.py – exporting history from RTC to a folder/file structure on a local disk (data/ folder)
  2. GIT_import.py – importing history from the local disk (data/ folder) into a Git repo

By separating export and import I am able to perform several trials for the export from RTC within redoing the import, and I can do import runs on all or a selection of the data without redoing the export. Note that an export or import may take several hours for large histories. I have done many export runs on selected components with a small history (so export is quick), and many imports on a (small) selection of exported data.

The data structure created by the RTC export looks like this:

data/rtc2git/changesets/<streamName>/<UUID>/meta.json
data/rtc2git/changesets/<streamName>/<UUID>/<path>/<file>
data/rtc2git/baselines/<streamName>/<UUID>/meta.json
data/rtc2git/snapshots/<streamName>/<UUID>/meta.json

The <UUID> is the unique identifier of the changesets, baseline or snapshot in RTC. The meta.json file contains meta-information such as the datetime stamp, deliver message, workitem, change type (e.g. “added”, “modified”, “renamed” or “deleted) and the <path>/<file> of the file or folder that has been changed by the changeset. Using the files on the local disk, the GIT import script is able to reconstruct the changeset into commits on the Git repository, which looks like:

rtc2git/.git/
rtc2git/<path>/<file>

As you know, the .git folder contains the Git repository and <path>/<file> represents the folders and files of the Git workspace. The Git workspace is (re)constructed using the exported changesets from RTC and then committed to the (local) Git repository. More details about GIT import in follow-up articles.

In the next article, I will dive into details about RTC export script.

About Frank Schophuizen (fschop)

Hi, my name is Frank Schophuizen and I am working as a consultant in CM, Agile and ALM for TOPIC Embedded Systems. I have over 30 years experience in software development in the technology industry, with the last 15 years mainly in process improvement, deployment and integration of methods and tools in the area of Agile (SAFe, Scrum), CM and ALM. I am strongly interested in the complexities of collaboration and integration in multi-project and multi-site organizations. I have worked with various technology companies such as Philips, ASML, NXP and Vanderlande, and with various tool vendors such as Atlassian (e.g. Jira, Confluence),IBM Rational (e.g. ClearCase, Synergy, Jazz products) as well as open source tools (e.g. SVN, Git, Jenkins, Trac, Eclipse). I am living in Eindhoven, the Netherlands, with my wife. We have 3 adult children. My main hobbies are classical music and photography.
This entry was posted in configuration management, git, migration, professional, rtc, tools. Bookmark the permalink.

2 Responses to RTC2GIT – Rational Team Concert to GIT migration (part 1)

  1. Pingback: RTC2GIT – Rational Team Concert to GIT migration (part 2) | fschop

  2. Pingback: RTC2GIT – Rational Team Concert to GIT migration (part 2) | fschop

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.