Team development in the Salesforce platform can be very frustrating if you don’t have the right process defined. For developers used to “traditional” software development it can be painful (it was for us), until you figure out the right way to do it.
Here at Soliant Consulting we have been working on defining the right process for quite some time, and it seems like finally we have most of the nuances figured out. We may not have all the answers, but we have a process that is working for us, and that is allowing us to keep our code under version control, and to automatically deploy to our Test/UAT environments through our Continuous Integration server.
Let’s start from the beginning. What are the main elements that you need to do team development in the Salesforce platform? They are really the same ones that you need in any other software platform. You need:
- A Version Control system (SVN, Git…)
- A separate Development Environment for each developer
- Unit Testing (Salesforce provides a JUnit-like framework)
- A Continuous Integration server (Jenkins/Hudson, Bamboo…)
As you can see, there is nothing new here. The way you use version control is also very similar to the way you would with any other development environment. This would be how you work with it (using SVN terminology).
Working with Version Control
See my Force.com IDE & SVN post for specifics on how to do these things with the Force.com IDE.
- You create a repository for your project
- One developer presumably will have a project that is connected to the Sandbox. This developer will commit all the code to the new repository we just created in step 1. This will be considered the first commit
- After that, every other developer in the project will check out the project from the repository. This will create a local project for each of them. Then each will give Force.com nature to his project, and will connect it to his development org (he will enter the Salesforce credentials for that org). When the Force.com IDE asks you if you'd like to Override/Refresh everything from server select “NO”
- Now that every developer has his own development environment they just need to do as usual and commit code to, and update from, the repository one or more times a day
What is important to keep in mind here is that (contrary to what you might have read in some of Salesforce’s documentation) you should treat the repo as the stable version of the code.
Repository = Gospel
Developer Org = Scratch Paper
In some of Salesforce’s documentation they talk about the org as containing the stable version of the code, but that is because they are referring to production or to a sandbox where all changes from production are being duplicated. We will see later how to address these issues in the context or our development process.
Let’s talk now about Continuous Integration. Some developers might not be as familiar with it as they are with Version Control and Unit Testing.
A CI server can do many things, but let’s list the main tasks will set it up to perform in the environment of our Salesforce Team Development process. On a scheduled basis (or every time a developer commits to the repo, depending how we set it up) the CI server will:
- Pull the code from the version control repository
- Deploy to our test/UAT environment (using the Ant Migration Tool)
- Notify the team by email if there was any issue deploying or if any of the tests failed
Steps 2 and 3 are sometimes in reverse order in other development environments. First the unit tests run, then the code is deployed, and finally functional testing is performed. With Salesforce however, you can only run the tests from inside an org, so we have to deploy first and then run our unit tests, followed by functional tests, if we have any.
Using Continuous Integration has many advantages, among others:
- Developers don’t need to be running the tests themselves every time they change code, if they don’t want to. Depending on the size of the team and the extent of the code change they might still want to do it anyway, but for small features/changes they can skip it since the CI server will run all of them and let them know if there was any failure
- There is no need to manually deploy to a test/UAT org for other stakeholders (project managers, business analysts, QA…) to see the status of the project. At all times the test/UAT org that we define will have the most up-to-date version of the code
- There is immediate feedback to the team if there is any issue with the code. Even if a developer committed some changes without updating, if those changes cause any conflict he (and everyone else in the team) will be immediately notified of the issue. This might also make developers be more careful when committing code
If this was a live presentation this would be the time where demoed our process. 🙂 Since it’s not, let me try to describe it.
Let’s assume that we already have our code under version control, and that we have two developers working on the project. Each of them (let’s call them John and Peter) has a separate development environment, and both are using the Force.com IDE. This could be a common set of events:
- Peter makes a change to his code in the IDE
- Force.com will automatically deploy this change to Peter’s development environment
- Peter commits the code to the repository
- Jenkins will detect the change in the repo and launch a build. The build will deploy to a separate org (what we’ve been referring to as our Test/UAT org) and will run all the tests there. Assuming there are no errors, no notification will be sent
- John updates his code from Subversion
- As the code is updated from the repo the Force.com IDE will automatically deploy the new changes to John’s development environment
Now everyone is in sync! You can go to the Test/UAT org and verify that Jenkins automatically deployed the changes there. They are now ready for QA.
Salesforce Development Scenarios
So far we’ve been talking about team development in general terms, and looking at the similarities it has with team development in other platforms. However, the Salesforce platform has some very distinct features, and we will now try to dive a little deeper into those.
There are actually two very distinct types of development in the Salesforce platform:
- Unpackaged applications
A. Developing a Salesforce Package
This is the easier of the two scenarios to deal with. When you are developing a package you are doing so in isolation from any other code. This means you don’t have to worry about not being able to install third-party managed packages in each of the developer’s orgs (like you do for unpackaged applications).
To do team development in this scenario you need the following:
- A version control repository for the project (as previously mentioned)
- One Developer Org per developer (they are free). Each of these should be connected to the repository (also previously mentioned)
- One Partner Developer Org for testing/QA. The difference between this org and the Developer Orgs is that in this case you can have up to 20 accounts, which will allow you to easily test with different profiles. (The orgs used for development could also be of this type, but there’s no need for it.) This org should also be connected to the repo.
- One Developer Org for Packaging, also connected to the version control repository. This is the org where the package will be created and uploaded to the AppExchange.
- Up to four orgs of different editions for User Acceptance Testing (UAT). These orgs will NOT be connected to the repo. You will install the package from the AppExchange to verify that the installation process also goes smoothly.
B. Developing a Salesforce Unpackaged Application
This scenario can be a bit trickier, as we’ll shortly see. To do team development in this scenario you need the following:
- A version control repository for the project (as before)
- One Developer Sandbox per developer (1 free in Professional Edition and 15 free in Unlimited Edition; more available for a fee). Each of these should be connected to the repository
- Possibly one Sandbox for internal testing (Developer Sandbox should be ok)
- Definitely one Sandbox for UAT (ideally a Full-copy Sandbox)
The main challenge here is that the Enterprise Edition (the most popular one) comes with only one Developer Sandbox. Additional ones need to be purchased from Salesforce. Only Unlimited Edition comes with multiple Developer Sandboxes.
In this scenario developers need to work with Developer Sandboxes because of the possibility of having third-party managed packages installed in production. If there were not managed packages in production (not with the characteristics described below) developers could actually use Developer Edition orgs. This will be described in more detail in the "Salesforce Unpackaged App Development Gotchas" section. Bear with me. We are almost there.
Salesforce Package Team Development Gotchas
There are a few important things to take into account when you are developing a Salesforce package with the methodology described in this article:
- You should NOT create the Package in any of the developer orgs. Creating a package will add the selected prefix to all classes, files, objects, etc., and may complicate things (more info regarding package creation)
- To avoid prefix issues so that your code can be deployed to multiple Developer Orgs, you should follow these recommendations:
- Do not use any prefix in Apex code (Apex code will work fine without the prefix even after the package has been created)
- Try to avoid dynamic SOQL. Since these queries are generated from text strings the namespaces won’t be automatically added. You could for example return Iterable<sObject> instead of Database.QueryLocator from the start method of classes that implement Database.Batchable. => This was an issue at some point in the global namespace, but it has been fixed
To automate some of these issues you may want to check out this Github project that adds namespace prefixes to the metadata source-code
Salesforce Unpackaged Application Team Development Gotchas
Why the Need for Developer Sandboxes
Let’s talk in more detail about a topic we’ve been avoiding so far. Why do we need to use Developer Sandboxes instead of Developer Edition Orgs for development? After all the latter ones are free. Well, the issue is that you need to be able to deploy code from one org to another. No only that, you need to be able to pull any new changes from production and deploy them to all of your development orgs. These changes might have dependencies to third-party managed packages.
Don’t get me wrong; we have nothing against managed packages. They provide great functionality at very reasonable prices (sometimes even free). The problem is that it is sometimes impossible to install a managed package in a Development Org. If you have managed packages installed in production and you cannot install those same packages in your development orgs you will never be able to use that org for development.
And why wouldn’t you be able to deploy code that you pulled from production into a Developer Org if there are managed packages installed in production? Because most likely (almost certainly) there will be references somewhere in the metadata to those managed packages. For example, a field might have been added in a layout that is actually a field of an object of the managed package. That field will have a prefix that does not exist in the Developer Org where you are trying to deploy your code.
Ok, but why wouldn’t I be able to install those packages in my Developer Orgs? Don’t package creators allow you to install the packages for free in a Developer Org? Unfortunately sometimes they do, and sometimes they don’t. There is for example a very popular managed packaged that cannot be installed in developer orgs. Another package of the same vendor can be installed with no problem. Another case that we have seen is that a client has a managed package installed in production that has now been deprecated. Which means that it’s just not possible to install it in any other org. However, if you have a Developer Sandbox all these packages will be automatically pulled from production, with the rest of the code, and installed.
A hack that I’m hesitant to bring up, but that might cross some developers minds, would be to try commenting out the sections of the code where there are references to the managed packages, but that’s most likely a recipe for disaster. It would mean that developers don’t really see the full picture. They would be testing their work in incomplete environments. It would also be very easy to forget commenting back in all those changes and end up removing functionality from production that you didn’t intent to remove.
So there you have it. If you are developing traditional Salesforce applications better provision a Developer Sandbox for each developer involved in the project. Otherwise robust team development will most likely not be possible.
Replicating Production Changes
Another very important issue to keep in mind is that all changes made in production must be replicated to all environments. If a change is part of the metadata, you should instruct the Salesforce admin (or whoever makes changes in production) to replicate the change in one specific Developer Sandbox. Once that change it’s in that Sandbox the Sandbox owner can pull it (use the synchronize perspective from the Force.com IDE to review it) and add it to the repo. At that point it will propagate to all of the developers’ environments. (If the change is not contained in the metadata I’m afraid it will need to be manually replicated to all development environments.)
Refreshing the Sandbox
Last thing to keep in mind in this section is that you should only refresh a Sandbox from production after all changes have been deployed. Never do that in the middle of an iteration or you might find yourself in a world of pain. Believe me, I know. 🙂
That’s all for now regarding team development in the Salesforce platform. Don't forget to take a look at the post regarding how to use the Force.com IDE with SVN. It can clarify questions you might have regarding the specifics of how to do some of the things described in this post. Also, I plan to write another post in the next few weeks regarding the Ant Migration Tool and how to use it with Jenkins. You may want to watch out for it. Until next time!