Bric::Hacker - A guide for Bricolage hackers.
$Revision: 1.49.2.1 $
$Date: 2004/05/06 03:59:53 $
This document is designed to provide information useful to Bricolage developers. If you've got questions about hacking Bricolage that aren't answered here please post to the Bricolage developer's mailing-list (see below) and tell us about it.
Bricolage has a number of mailing-lists that are relevant to developers:
http://lists.sourceforge.net/lists/listinfo/bricolage-general
http://lists.sourceforge.net/lists/listinfo/bricolage-commits
You can often find folks hanging out and occasionally discussing development issues on the #bricolage channel on the Rhizomatic IRC network.
If you're developing Bricolage then you should be working with the latest code from the Bricolage Subversion repository. You can browse the Subversion repository at http://svn.bricolage.cc/viewcvs/.
Information on connecting to the repository to checkout a working copy is at http://www.bricolage.cc/svn.html.
Bricolage has a Bugzilla server dedicated to it:
http://bugzilla.bricolage.cc/
You should use this system to report bugs in Bricolage. If you're looking for something to do you can also use the system to find open bugs and fix them. For more things to do see Bric::ToDo.
Patches should be generated using the svn diff command on each of the
files modified, from the root directory. For example, if you made changes to
lib/Bric/Changes.pod and comp/foo.mc, you would generate a diff by
running this command from the root of your Subversion checkout:
svn diff lib/Bric/Changes.pod comp/foo.mc > patch.txt
If you created one or more new files in your changes then you'll have to add
them to the patch separately using normal diff against /dev/null. For
example, if you created the file inst/upgrade/1.9.1/solve_fermat.pl then
you would add this to patch.txt with:
diff -u /dev/null inst/upgrade/1.9.1/solve_fermat.pl >> patch.txt
Always create patches using an up-to-date Subversion checkout if possible. Send your patches to the bricolage-devel list mentioned above.
Patches created using the method above can be applied using patch with the
-p0 option from the root of your Subvesion checkout:
patch -p0 < patch.txt
Make sure you check the results with svn diff before committing.
Try to follow the style of the existing Bricolage code. Except where it is bad, of course. Basically, write in the style you see in most Perl books and documentation, particularly perlstyle.
Although historically the Bricolage code has not enforced whitespace rules, we now request that you use 4-space indents (2 spaces for continued lines) and discourage the use of tabs. The following settings for some of the more popular editors are thus recommended while editing Bricolage source code.
We strongly recommend that you use cperl-mode while editing Bricolage
sources in Emacs. Grab the latest version from the CPAN, install it, and then
place the following in your ~/.emacs file:
(custom-set-variables '(case-fold-search t) '(cperl-indent-level 4) '(cperl-continued-statement-offset 2) '(cperl-tab-always-indent t) '(indent-tabs-mode nil)) ;; let hashes indent normally; I think this requires ;; at least version 4.32 of cperl-mode.el '(cperl-indent-parens-as-block t) '(cperl-close-paren-offset -4)
Also, if you'd like to take advantage of the full functionality of
cperl-mode and have it automatically parse all Perl source files, add these
settings, as well:
(defalias 'perl-mode 'cperl-mode)
(setq auto-mode-alist
(append
'(("\\.\\([pP]\\([Llm]\\|erl\\)\\|al\\|pod\\)\\'" . cperl-mode))
auto-mode-alist))
(setq cperl-hairy t)
(setq interpreter-mode-alist (append interpreter-mode-alist
'(("miniperl" . cperl-mode))))
When editing Bricolage Mason components, mmm-mode can help. It's Mason mode
will parse Mason component files and use sgml-mode in HTML spaces and
cperl-mode in Mason blocks. Grab it from
http://mmm-mode.sourceforge.net/, install it, and then add the following to
your ~/.emacs file to have it automatically parse your Bricolage Mason
component files:
(add-to-list 'load-path "/usr/local/share/emacs/site-lisp")
(require 'mmm-mode)
(require 'mmm-mason)
(setq mmm-global-mode 'maybe)
(add-to-list 'auto-mode-alist '("/usr/local/bricolage/comp" . sgml-mode))
(mmm-add-mode-ext-class 'sgml-mode "/usr/local/bricolage/comp" 'mason)
And if you need to examine the Mason object files created by Bricolage in
order to chase down bugs and such, you can use the cperl-mode in those
files by adding this to your ~/.emacs file:
(add-to-list 'auto-mode-alist '("/usr/local/bricolage/data/obj" . cperl-mode))
Rafael Garcia-Suarez has written a Vim indent macro which (for the most
part) duplicates the behavior of Emacs perl-mode. It is now, as of Vim
6.0, included in the Vim distribution and should be found in
$VIMRUNTIME/indent/perl. The easiest way to use it though is to place the
following line in your .vimrc:
source $VIMRUNTIME/indent.vim
You'll also need to add these lines.
set tabstop=8
set softtabstop=4
set shiftwidth=4
set expandtab
The first three lines make Vim duplicate the behavior of Emacs in creating the appearance of 4 space tabs with a mix of tabs and spaces. This is necessary for reading older Bricolage files which were written this way, and haven't yet been re-tabbed.
The expandtab setting does the Right Thing under the new rules, in that it doesn't use tabs at all, only spaces.
The standard for writing SQL in Bricolage is pretty straight-forward: format the SQL in 80 columns or less, and use heredocs where possible. For example:
my $sql = "<< END_SQL";
SELECT id, name, description, publish_date, cover_date, current_version,
publish_status, active
FROM story
WHERE id = ?
END_SQL
Also, when writing your queries, please follow the following rules with regard to table aliases:
The reason for these rules is that single-letter aliases for queries are as unreadable and unmaintainable as single-letter variables names. You get halfway down the page, and you can no longer remember to what tables they refer.
The rules above are acutally taken from O'Reilly's ``Introduction to PL/SQL Programming,'' which has an excellent chapter on code cleanliness in SQL and SQL-extension languages.
Keep subroutines short. Each subroutine should handle one task. For example,
if you have a subroutine get_foo, and getting ``foo'' requires getting ``bar''
and ``qux'', then make two more subroutines get_bar and get_qux and put
them in the ``private functions'' section. Also if you duplicate some code in
several subroutines, factor it out into another subroutine. It's easier to
maintain one subroutine than to maintain three.
Limit the width of your code to 80 characters if possible. This makes them easier to read. Keeping subroutines short helps to meet this goal, as well.
Make enough comments so that someone maintaining your code can understand what is going on. Comments shouldn't be redundant with the code. They should explain non-obvious code. Comments can be bad in some cases if code changes and the corresponding comment is not kept in sync. So keep the comments in sync!
Use Perl idioms if it is clear and concise, but use them with care. Implicit variables can be slick, but hard to understand; add a comment if you use uncommon ones.
Use Mason components only for display, and put business logic into library modules and callback components. But only for as long as callbacks are in components! We hope to move them into libraries soon, at which time this rule will become even more rigid.
For POD, generally you copy/paste it from another module. Recently
we are trying to cut out some of useless things like
B<Side Effects:> NONE. For a canonical example, see
Bric::Biz::Site.
Avoid magic values, i.e. don't hard-code numbers into code. It tends to become a maintenance problem if you put the number 1023 throughout a module and then somewhere else you have 1021; is 1021 related to 1023 (1023 - 2), or is it just another random number? And what does 1023 mean? Put a constant at the top of the module and use that, instead.
In the end, just try to follow the existing code style, and take into consideration any feedback you get from the mailing list when patches are submitted.
All Bricolage patches should be accompanied by the necessary tests. You are strongly encouraged to write comprehensive tests that thoroughly test whatever changes or additions you make to the Bricolage API. You are also strongly encouraged to write tests for the current API, if you find that adequate tests have not yet been written (and this is true in a great many places, unfortunately). Although Bricolage does not yet support UI tests, we take our API testing seriously, and so should you. Here's what you need to know to write tests for Bricolage.
Bricolage contains a test suite based on Test::Class. The test classes live in the t/Bric directory, and all subclass Bric::Test::Base. If you're familiar with Test::More, then the syntax for how the Test::Class-based classes work should be pretty readily apparent. See the Test::Class documentation for details. It's definitely worth a read. If you haven't used Test::More, its documentation is also a must-read.
Bricolage has two different sets of tests, those run by make test and those
run by make devtest. The former are stand-alone tests that don't rely on the
presence of a Bricolage database to be run. They can thus be run before
make install. The idea here is that we offer a basic set of tests that can
be run via the standard Perl installation pattern of
perl Makefile.PL make make test make install
The tests run by <make devtest> are intended to be run during development.
They require that a Bricolage database be installed and running. They will
SELECT, INSERT, UPDATE, and DELETE data from that database, so
make devtest should never be run against a production database. Indeed,
you should in general have a clean, fresh database installation to run the
tests against. Although the test suite makes every effort to clean up after
itself by DELETEing data it has added to the database, it is unfortunately
imperfect, and extra data will be left, especially in Group-related and
Attribute-related tables.
So, once more, make devtest should never be run against a production
database. 'Nuff said.
You can get a clean database without reinstalling Bricolage by doing
sudo make db sudo make db_grant
Answer yes when it asks you to drop the database. Then to get useful debugging output, do verbose testing with something like
sudo make devtest TEST_VERBOSE=1 > test.out 2>&1
All of the tests are run by the inst/runtests.pl script, which in turn
tells Test::Harness to execute t/Bric/Test/Runner.pm. This
file will find all the necessary test classes, load them, and then run their
tests. inst/runtests.pl takes a number of arguments to simplify the running
of scripts, including a list of test files or classes to run, so that you can
just run the tests you need to run while you're developing new tests.
perldoc inst/runtests.pl for more information.
Be sure to use the testing base classes in your test classes. For non-database dependent tests (which are always in files named Test.pm, use Bric::Test::Base. For development tests, use Bric::Test::DevBase, which inherits from Bric::Test::Base. Be sure to read the documentation in these classes, as it will help you to write your tests more effectively. Pay special attention to the methods added to Bric::Test::DevBase, as they're there to help you clean up any new records you've added to the database.
And finally, when you do provide patches to Bricolage, along with the new
tests to test them, make sure that all existing tests pass, as well. This
means that you should always run make devtest on a fresh database build
with your changes. Furthermore, if your patch involves changes to the
database, you should provide the necessary upgrade script in inst/upgrade/,
and also run the tests against a database that has been built from Bricolage
sources untouched by your patch, and then upgraded by your upgrade
script. This will help to ensure that your upgrade script modifies the
database in the same way as your patch modifies the Bricolage SQL files.
And with that said, happy testing!
Bricolage is a complex application and debugging can be difficult. Here are some tips to help you find bugs faster:
QA_MODE in your bricolage.conf file. This will cause the UI to
display a bunch of useful data at the bottom of every page, including session
state, cache state, cookies, and GET and POST args. QA_MODE turns on
PerlWarn (use warnings) and causes error messages to include more
information. QA_MODE also enables Apache::Status at the URL /perl-status
which allows you can to examine the state of the Perl interpreter inside
Apache.
Set the NO_TOOLBAR directive to ``0'' or ``Off'' in bricolage.conf to keep
Bricolage from popping up a new browser window without a toolbar.
Run Bricolage under the Perl debugger using the debug command with
bric_apachectl:
bric_apachectl debug
This will start Apache in single-process mode (httpd -X) and setup
Apache::DB to start the debugger on the each hit to the server. You'll need to
install the Apache::DB Perl module to use this command.
To run Bricolage under DDD (http://www.gnu.org/software/ddd/), start ddd as
root and load bin/bric_apachectl. Give it the argument ``debug'' and run
it. When you issue a hit to the server the debugger will stop on the first
line of Bric::App::Handler::handler(). From there you can set breakpoints
inside Bricolage and debug normally.
If you prefer to run without the debugger using only the Apache single-process mode, then run Bricolage using the command
bric_apachectl singleSet
DBI_DEBUG and DBI_CALL_TRACE to 1 in your bricolage.conf
file. DBI_DEBUG records every database call in the logs complete with SQL
and arguments. DBI_CALL_TRACE adds a a subroutine call trace for each
statement showing where the database call originated. This generates a lot of
data but it can be very helpful.
Look at the database directly using psql. Many bugs in Bricolage can only
be successfully diagnosed by examining records created in the database.
Bricolage has two separate profiling systems that you can use to extract performance data:
PROFILE variable
on in your bricolage.conf file and restart your server. This will create a
profiler/$$/tmon.out file in your Apache log directory for each server
process. You can use dprofpp to analyze these files after the server has
been stopped with bric_apachectl stop. See the Devel::Profiler
documentation for more details.
The database profiler is activated by enabling the DBI_PROFILE option in
bricolage.conf. This causes database profiling traces to be written to the
Apache error log during requests. You can then use bric_dbprof to analyze
these trace. See bric_dbprof for details.
CAUTION: Neither of these options is appropriate for a production system.
A relatively simple way to contribute to Bricolage is to provide actions and movers. These are plugin modules that can add new functionality to Bricolage without needing to make changes to the existing API.
An ``action'' is an act that is performed on files before they are distributed. Say you want to clean the HTML of all of your HTML files before they're distributed. You'll need to create an action to do this. Consult the Bric::Dist::Action manpage for information on how to create actions.
Say you need to distribute files via a protocol that Bricolage doesn't currently support -- say, an new variant of FTP called ``FooTP.'' You'll need to create a new mover. Consult the Bric::Dist::Action::Mover manpage for details on how to do that.
If you're a Bricolage developer with permission to commit to the Subversion repository, you may occasionally have to merge changes from a release branch (where bug fixes are generally committed) into the trunk. Here's how to do it with a minimum of hassle.
svn status will tell
you if there have been any uncommitted changes to the local copy of the branch
or trunk. Running svn update will merge in any updates from the copy on the
Subversion server. So we recommend runing both of these commands in both the
branch and the trunk and committing any necessary chaanges before continuing.
% cd bricolage-trunk % svn status % svn update At revision 6123.
% cd bricolage-branch-checkout % svn status % svn update At revision 6123.To merge the changes from the branch into the trunk, you need to tell the Subversion <merge> command to use the difference between two revisions in the branch to update the trunk. The two revisions to compare are, on the one hand, either the revision when the branch was created, or the revision when the branch was last merged into the trunk; and on the other hand, the current revision in the branch.
For the latter number, you can simply use the HEAD keyword, which
represents the latest revision number in the branch.
For the former number, Bricolage has implemented a standard way of recording a merge: we create a new tag for the branch. In Subversion, branches and tags are exactly the same, the difference is only in convention. So the convention for Bricolage is to store tags in the tags subdirectory of the project directory, and branches in the branches subdirectory. The other half of the convention is that no one should ever commit changes to a tag after it has been created.
Tags are named for the branch name, the word ``merge'', and the revision number up to which the merge was made. For example, if a merge was committed to the trunk from the rev_1_8 branch in Subversion revision # 5694, the branch would be tagged ``rev_1_8_merge_5694''.
So get the revision number of the last merge, look in the Bricolage ViewCVS
interface for the last merge tag created for the branch. Or use the Subversion
list command to find it:
% svn list http://svn.bricolage.cc/bricolage/tags | grep rev_1_8 rev_1_8_merge_5694 rev_1_8_merge_5825 rev_1_8_merge_5902 rev_1_8_merge_5995
So now we know that we want to update the trunk with all of the changes
between revision 5995 and HEAD. If no merge tag exists, then use svn log
to find out at what revision number the branch was created and use that
number.
% cd bricolage-trunk % svn merge -r 5995:HEAD http://svn.bricolage.cc/bricolage/branches/rev_1_8 U lib/Bric/App/ApacheConfig.pm C lib/Bric/Hacker.pod
If you're nervous about doing the merge (don't be, it's just your local copy
you're updating at this point), you can use the --dry-run option to see
what files would change without changing them. Or run svn diff with the
same arguments to see what the differences are:
% svn merge -r 5995:HEAD http://svn.bricolage.cc/bricolage/branches/rev_1_8 \
> test.diff
Resolve any conflicts (hopefully none, but they do happen occasionally).
Notice above that the result of the merge is a conflict in
lib/Bric/Hacker.pod. To resolve the conflicts, follow the directions in the
Subversion book, published online here:
http://svnbook.red-bean.com/svnbook/ch03s05.html#svn-ch-3-sect-4.4.
Rebuild the database and run all of the tests:
% make devtest TEST_VERBOSE=1
If there are any test failures, you'll need to fix those, too.
Commit the changes to the trunk. Be sure to specify what revisions were merged in, and what the new tag you'll create in the branch will be named. The final revision number (represented by HEAD) happened to have been printed out when we ransvn update in step one, so it's easy to find and use:
% svn commit -m 'Merged rev_1_8 changes 5995-6123. Will tag rev_1_8_merge_6123.'Tag the branch with the last revision, so that future merges can decide to merge only from thiat revision on.
% svn cp http://svn.bricolage.cc/bricolage/branches/rev_1_8 \
http://svn.bricolage.cc/bricolage/tags/rev_1_8_merge_6123 \
-m 'Merge tag for rev_1_8 to revision 6123.'
By following this methodology we should be able to minimize the number of conflicts we get between merges. See book ``Version Control with Subversion,'' chapter 4 at http://svnbook.red-bean.com/svnbook/ch04.html, for a more detailed explanation of the whys and wheres of this approach to merging Subversion branches.
Note: To be updated to reflect use of Subvesion!
If you are a Bricolage release manager and you're getting ready to release a new version, here are the steps you'll need to take to create a distribution tarball. First, proof-read the list of changes in Bric::Changes. Also, be sure to add today's date to the header for the new release:
=head1 VERSION 1.x.x (2003-11-29)
Next, tag the release in CVS:
cvs -qz3 tag rel_1_x_x
If this is a major release, you'll need to create a new branch, so that it can be maintained for bug fixes separately, and then tag the release in that branch.
cvs -qz3 tag -b rev_1_x cvs -qz3 tag rel_1_x_x
Next, export the sources into a new directory. Be sure to use the -kkv
option with export so that the version numbers are all properly populated.
cd /tmp cvs -qz3 export -r rel_1_x_x -kkv bricolage cd bricolage make dist
This will create a distribution tarball in the bricolage directory. Copy this tarball somewhere, and do a full test with it, to make sure that Bricolage does indeed build and properly install itself.
cp bricolage-1.x.x.tar.gz /tmp/src cd /tmp/src tar zxvf bricolage-1.x.x.tar.gz cd bricolage-1.x.x perl Makefile.PL make # Shut down PostgreSQL. make test # Start PostgreSQL. sudo make install make devtest sudo bric_apachectl start # Log in to the UI and test it a bit. sudo make uninstall
Be sure to run make devtest after everything is installed to make sure that
it is all functioning correctly. Users won't be running these tests, as they
muck up the database. But that's okay for our validation of the release
tarball. Use make uninstall to clean up the mess. If you have to go back
and make any changes, be sure to update the tag of any files you change and
commit to CVS, then do the whole thing over again. Once everything appears to
be working properly, release!
Sam Tregar <stregar@about-inc.com>
David Wheeler <david@wheeler.net>