So what is it about bug tracking systems? Some people claim they don't need
one, others hate them and yet others can't live without them! Yet whether
they realize it or not, all development activities use a bug tracking system
of some sort, whether it's a text file, spreadsheet, home grown solution or
commercial product. Further, any software development company producing
quality software needs one to monitor the improvement in quality of their
product as the release date approaches, manage enhancement requests, allow
customers to track the progress of their requests or bug reports, etc.
There is no end of different types of bug tracking systems available.
However, some are obviously better and more suitable than others, especially
as the size and scope of the project or company grows. The following
sections describe the main categories than most bug tracking systems fall
into.
Text File
At the very basic level you've got a text file solution, the classic
"bugs.txt". This is probably what every small development effort starts with
(once they've got past the collection of sticky notes phase - but we're not
even going to go there !). This is a good starting point as it shows the
problem of bug tracking has been acknowledged - and the key to solving any
problem, is first admitting it. However, this solution starts to fail very
quickly as there is no scope to prioritize entries, monitor their status,
maintain any history, etc. Once the size of the development team is more
than one, you then run into the questions of who's working on what and file
locking issues. So, not really a solution for anything more than a single
developer working on a private project.
Spreadsheet
Next comes the spreadsheet solution. A step up from a text file as you now
have more scope to add columns such as priority, status, assignee and
resolution. You still have file locking issues though. What happens when
Developer Dan opens the spreadsheet and goes to lunch or goes home for the
day? - no-one else can use the system. You can't very well say to a customer
"I'm sorry I don't know the status of your bug as I can't access the bug
tracking system", not a good impression and you probably won't keep your
customers very long.
MS Access
The next step up is a simple MS Access database solution. This has all the
same advantages as a spreadsheet with the added benefit of being able to
introduce a "pretty interface" through the use of forms. However, in its
simplest (and most common) form you are still left with a single user (or at
least one user at a time) solution. This solution doesn't extend to cross
platform development and has the added constraint of requiring all users to
run MS Access.
Home Grown
Stepping up another notch takes you to a fully fledged home grown solution.
Obviously there are no end of forms that this could take and will generally
depend on the in-house expertise available, eg., an Oracle shop would base
their solution on an Oracle database, whereas a Windows shop might implement
a VB client solution. While there's no arguing that you'll get a solution
totally customized to your needs, it's all at a price, the old "buy vs build"
argument. So the question comes down to whether you want to be in the
business of building a bug tracking system for internal use or building XYZ
for selling to your customers? After all why reinvent the wheel?
3rd Party
Finally we reach the 3rd party solution, this can be either open source or
commercial. While there's no denying the merits of open source, it all comes
down to how much time and effort (and therefore money) is required to install
and maintain your chosen solution. There's no point in selecting a solution
because it's "free" only to have to become a expert in 10 other packages that
it is dependent on, just to use it! This puts you right back in the position
of the homegrown solution. Similarly, it's not a good idea to select a
solution that requires a database if you have no in-house database experience,
otherwise you'll encounter problems on installation, backup of your data,
moving to a different server, etc. In general your selection should reflect
your development environment so that you can leverage off existing expertise
for ease of installation and ongoing administration. However, you should
still be looking for a solution that requires minimal installation effort and
will run out of the box, yet allow customization to suit your corporate
environment and needs.
The following list is a summary of the main features that should be
considered when making your bug tracking system selection. Not all features
may be required and some will rate higher than others depending on your
specific requirements.
Cross platform or OS specific.
Do you need to run the server on multiple
platforms or are you tied to a specific OS? Might you change platforms
from Windows to Linux, for example?
Web based or fat client.
Do you need to support clients on multiple
platforms? Do you want to install clients everywhere access is required?
Do you need to support remote developers or customers?
Database or file system backed.
Do you have the in-house expertise to
support a database solution or does a file based repository better suit
your needs?
Access control.
Can access to the bug tracking system be controlled to the
granularity you require? Eg., on a role, user, module, bug basis.
Customizable reference data.
Can the values for supplied reference data
(eg., priority and status) be modified to suit your corporate naming
conventions? Can additional fields be added to allow tracking of module
specific attributes?
Customizable interface.
Can the look and feel (eg., colours, logos) of the
user interface be customized to reflect your corporate standards?
Searchable.
Can the bug repository be searched and/or allow filtering on
specific criteria such as assignee, status or priority?
Reporting.
Can reports be produced from the bug repository? Is the output
format of the reports suitable for printing or importing into other tools
(eg., spreadsheet) for further analysis?
Email notifications.
Can users be notified by email when a bug is changed?
Can the format of the notifications be customized?
Unlimited number of users, bugs or projects.
Does the terms of the license
place any constraints on the number of users, etc?
Auditable.
Is an audit trail produced reflecting all changes to a bug?
SCM integration (source code control).
Is integration with your SCM system
(eg., CVS, PVCS, SourceSafe) available?
IDE integration.
Is integration with your IDE (eg., Netbeans/Forte,
Eclipse, JBuilder) available?
Internationalization support.
Is the input of non-ascii characters (eg.,
accented characters) and multi-byte characters (eg., Japanese) supported?
Can the user interface be displayed in languages other than English?
Dependency on other packages.
What packages is the bug tracking system
dependent on? Do you have in-house expertise in these packages?
Ease of installation.
Can the bug tracking system "run out of the box" or
does it require significant customization?
File attachment support.
Can files (eg., screen shots, log files) be
attached to the bugs?
Cost.
Always important but shouldn't be the primary consideration. First
and foremost make sure the bug tracking system will meet your technical and
business requirements.
The above system types and considerations can be used to form
the basis of the requirements for a solution to suit the specific
needs of your environment.
This list of requirements can then be used as the selection criteria for
evaluating the range of bug tracking systems available.
Jill Stephenson is one of the founders of
Tortuga Technologies, a software
development company based in Brisbane, Australia. She has over 15 years
experience across a wide range of companies in the UK, USA and Australia.
She is currently involved in the development of
Ozibug, a web based bug tracking system.
Ozibug is implemented as a Java servlet allowing it to be run on any platform
for which a JVM and servlet container is available. In keeping with the
spirit of quality software development, each version of Ozibug is tested on
over 50 Servlet Container/JVM/Operating System combinations before release.
