Bedrock

playdoh Documentation
Release 1.0
Mozilla
January 15, 2014
Contents
1 Contents 3
1.1 Installing Bedrock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Vagrant Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 Installing and Learning About the PHP Site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.4 Localization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.5 Developing on Bedrock . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.6 How to contribute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
1.7 Newsletters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.8 Tabzilla . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
i
ii
playdoh Documentation, Release 1.0
bedrock is the code name of the new mozilla.org. It is bound to be as shiny, awesome, and open sourcy as always.
Perhaps even a little more.
bedrock is a web application based on Django/Playdoh.
Patches are welcome! Feel free to fork and contribute to this project on Github.
Contents 1
2 Contents
CHAPTER 1
Contents
1.1 Installing Bedrock
1.1.1 Installation
Its a simple Playdoh instance, which is a Django project.
These instructions assume you have git and pip installed. If you dont have pip installed, you can install it with
easy_install pip.
Start by getting the source:
$ git clone --recursive git://github.com/mozilla/bedrock.git
$ cd bedrock
(Make sure you use recursive)
Important: Because Bedrock uses submodules, it is important not only to use --recursive on the initial clone,
but every time you checkout a different branch, to update the submodules with:
git submodule update --init --recursive
You might want to create a post-checkout hook to do that every time automatically, by putting that command in a le
bedrock/.git/hooks/post-checkout.
You need to create a virtual environment for Python libraries. Skip the rst instruction if you already have virtualenv
installed:
$ pip install virtualenv # installs virtualenv, skip if already have it
$ virtualenv venv # create a virtual env in the folder venv
$ source venv/bin/activate # activate the virtual env
$ pip install -r requirements/compiled.txt # installs compiled dependencies
$ pip install -r requirements/dev.txt # installs test dependencies
If you are on OSX and some of the compiled dependencies fails to compile, try explicitly setting the arch ags and try
again:
$ export ARCHFLAGS="-arch i386 -arch x86_64"
$ pip install -r requirements/compiled.txt
Now congure the application to run locally by creating your local settings le:
3
$ cp bedrock/settings/local.py-dist bedrock/settings/local.py
You shouldnt need to customize anything in there yet.
Check out the latest product-details:
$ ./manage.py update_product_details
This pulls in version information for Mozilla products like Firefox.
Lastly, you need to install node and the less package. Soon you wont need this for local development but currently it
compiles the LESS CSS code on the server-side:
$ npm -g install less
You dont have to use npm to install less; feel free to install it however you want.
Add the path to the LESS compiler (found by using which lessc) to bedrock/settings/local.py with the following line:
LESS_BIN = /path/to/lessc
1.1.2 Make it run
To make the server run, make sure you are inside a virtualenv, and then run the server:
$ ./manage.py runserver
If you are not inside a virtualenv, you can activate it by doing:
$ source venv/bin/activate
If you get the error NoneType is not iterable, you didnt check out the latest product-details. See the above section
for that.
1.1.3 Run it with the whole site
If you need to run the whole site locally, youll need to rst set up the PHP side, and then also set up to serve Bedrock
from the same Apache server at /b/. Thats because the rewrite rules in the PHP and Apache cong assume they can
serve requests from Bedrock by rewriting them internally to have a /b/ on the front of their URLs.
Important: Before continuing, go get the PHP side working. Then come back here.
One way to add Bedrock to your local site, once you have the PHP side working, is to use runserver to serve Bedrock
at port 8000 as above, then proxy to it from Apache. The whole virtual server cong might end up looking like this:
<VirtualHost
*
:80>
ServerName mozilla.local
VirtualDocumentRoot "/path/to/mozilla.com"
RewriteEngine On
RewriteOptions Inherit
ProxyPass /b http://localhost:8000
ProxyPassReverse /b http://localhost:8000
ProxyPass /media http://localhost:8000/media
ProxyPassReverse /media http://localhost:8000/media
Include /path/to/bedrock/etc/httpd/global.conf
</VirtualHost>
4 Chapter 1. Contents
But you might have better success using a real WSGI setup that is closer to what the real servers use. The following
conguration is simplied from what the bedrock staging server uses.
Assumptions:
A Red Hat or Debian-based Linux distribution. (Other distributions might not have Apache HTTP Server
installed and congured the same way.)
Apache HTTP Server with php and mod_wsgi
Subversion mozilla.com checkout at /path/to/mozilla/mozilla.com
Subversion mozilla.org checkout at /path/to/mozilla/mozilla.com/org (ideally as an SVN external)
Bedrock checkout at /path/to/mozilla/bedrock
Create a local cong les for mozilla.com and mozilla.org:
$ cp /path/to/mozilla.com/includes/config.inc.php-dist /path/to/mozilla.com/includes/config.inc.php
$ cp /path/to/mozilla.com/org/includes/config.inc.php-dist /path/to/mozilla.com/org/includes/config.inc.php
Edit /etc/hosts and add:
127.0.0.1 mozilla.local
Apache cong - create le /etc/apache2/sites-available/mozilla.com:
# Main site at /, django-bedrock at /b
<VirtualHost
*
:80
*
:81>
ServerAdmin user@example.com
DocumentRoot "/path/to/mozilla/mozilla.com"
AddType application/x-httpd-php .php .html
DirectoryIndex index.php index.html
RewriteEngine On
<Directory "/path/to/mozilla.com">
Options MultiViews FollowSymLinks -Indexes
AllowOverride All
</Directory>
RewriteMap org-urls-410 txt:/path/to/mozilla.com/org-urls-410.txt
# In the path below, update "python2.6" to whatever version of python2 is provided.
WSGIDaemonProcess bedrock_local python-path=/path/to/bedrock:/path/to/venv-for-bedrock/lib/python2.6/site-packages
WSGIProcessGroup bedrock_local
WSGIScriptAlias /b /path/to/bedrock/wsgi/playdoh.wsgi process-group=bedrock_local application-group=bedrock_local
Alias /media /path/to/bedrock/media
<Directory /path/to/bedrock/media>
AllowOverride FileInfo Indexes
</Directory>
Include /path/to/bedrock/etc/httpd/global.conf
</VirtualHost>
Then enable the new site, build the css and js les, and nally restart apache:
sudo a2ensite mozilla.com
sudo a2enmod expires headers actions
1.1. Installing Bedrock 5
python manage.py compress_assets
sudo service apache2 restart
Troubleshooting
If you get Django error pages reporting I/O errors for .css les, its because not all the .css les were compiled before
starting Apache and Apache does not have write permissions in the media directories. Running python manage.py
compress_assets should solve it. Remember to run that command again anytime the css or less les change.
If you change Python les, either restart Apache or touch playdoh.wsgi, so that the WSGI processes will be restarted
and start running the new code.
If youre working on the rewrite rules in bedrock/etc/httpd/
*
.conf, be sure to restart Apache after any
change. Apache doesnt re-read those les after it has started.
1.1.4 Localization
If you want to install localizations, just check out the locale directory:
git svn clone https://svn.mozilla.org/projects/mozilla.com/trunk/locales/ locale
# or
svn checkout https://svn.mozilla.org/projects/mozilla.com/trunk/locales/ locale
You can use git or svn to checkout the repo. Make sure that it is named locale. If you already have it checked out
as locales, just do:
ln -s locales locale
You can read more details about how to localize content here.
1.1.5 Upgrading
On May 15th, 2013 we upgraded to a newer version of Playdoh. This brought with it a lot of structural changes to the
code. Here are the required steps to get up and running again with the latest code:
# get the code
git pull origin master
# update the submodules
git submodule update --init --recursive
# move your local settings file
mv settings/local.py bedrock/settings/local.py
# remove old empty directories
rm -rf apps
rm -rf settings
rm -rf vendor-local/src/django
rm -rf vendor-local/src/tower
rm -rf vendor-local/src/jingo-minify
That should do it. If youre not able to run the tests at that point (python manage.py test) then there are a
couple more things to try.
1. If you have a line like from settings.base import
*
in your bedrock/settings/local.py
le, remove it.
2. If you were setting a logging level in your bedrock/settings/local.py le, you may now need to
explicitly need to import it (import logging).
Otherwise please pop into our IRC channel (#www on irc.mozilla.org) and well be happy to help.
1.1.6 Notes
A shortcut for activating virtual envs in zsh is . venv/bin/activate. The dot is the same as source.
Theres a project called virtualenvwrapper that provides a better interface for managing/activating virtual envs, so you
can use that if you want.
1.2 Vagrant Installation
The Vagrant installation will help you work on the Python bedrock codebase and the PHP legacy codebase with a
minimum amount of effort (hopefully).
This entire process will take between 30-50 minutes. For most of this time you will not be doing anything, Vagrant
will be automagically downloading and conguring. This is a good time to have a cup of tea and/or coffee, possibly
walk the dog.
1.2.1 Preparing Your System
1. Install Vagrant.
Vagrant is a manager of VMs for development.
Based on a conguration le, Vagrant will create a Virtual Machine, downloading and conguring
everything you need to have a local environment running.
Visit Vagrants download page.
This installation is tested using version: v1.2.5
Do not install via apt-get, the version installed (in debian wheezy) appears broken.
2. Install Virtualbox.
You are required to have virtualbox installed. This can be downloaded at the virtualbox download
page.
For Debian based systems:
~$ sudo apt-get install virtualbox
3. Install git.
The bedrock code is revisioned using git <http://git-scm.org>.
~$ sudo apt-get install git
For other Linux distributions or operating systems visit Gits download page.
4. Install svn.
The legacy php code is revisioned using SVN.
1.2. Vagrant Installation 7
~$ sudo apt-get install subversion
For other Linux distributions or operating systems visit SVNs download page.
1.2.2 Build The Environment
1. Directory Setup.
Create a top level directory to hold both bedrock and the legacy le system. You could call this
directory bedrock-legacy. The following steps take place under that directory.
2. Using Git Clone Bedrock Repository.
Bedrock is hosted at http://github.com/mozilla/bedrock.
Clone the repository locally:
~bedrock-legacy$ git clone --recursive http://github.com/mozilla/bedrock
Note: Make sure you use --recursive when checking the repo out! If you didnt, you can load
all the submodules with git submodule update --init --recursive.
3. Using SVN Checkout The Locale Repository. (Optional)
If you would like to see localized versions of the site you will need to checkout the locale directory
to the root of the bedrock directory you just cloned.
~$ cd bedrock
~bedrock-legacy/bedrock$ svn checkout https://svn.mozilla.org/projects/mozilla.com/trunk/locales/ locale
Note: You can read more details about how to localize content here.
4. Using SVN Checkout Mozilla.com PHP Repository.
Mozilla.com PHP is hosted on https://svn.mozilla.org/projects/mozilla.com/trunk.
~bedrock-legacy$ svn co https://svn.mozilla.org/projects/mozilla.com/trunk mozilla.com
Note: At this stage you should have two directories side-by-side. bedrock and mozilla.com.
1.2.3 Congure The Environment
1. Congure Bedrock.
Congure Bedrock by creating and editing the local settings le:
~bedrock-legacy$ cp bedrock/bedrock/settings/local.py-dist bedrock/bedrock/settings/local.py
Add this line below LESS_PREPROCESS:
LESS_BIN = /usr/local/bin/lessc
2. Congure Mozilla PHP.
Congure the legacy site by creating and editing the local settings le:
cd mozilla.com/includes
cp config.inc.php-dist config.inc.php
Set the following values:
$config[server_name] = local.mozilla.org;
$config[file_root] = /srv/legacy;
3. Set A Host Name.
We need to set a host name that you will use to access vagrant from a web-browser. You will need to
add the following to your hosts le (note you may need sudo permissions).
192.168.10.55 local.mozilla.org
The hosts le can be found in the following directories.
For Debian & OS X based systems:
/etc/hosts
For Windows based systems
c:\windows\system32\drivers\etc\hosts
1.2.4 Start Your Machine
1. Fire up vagrant.
Now you need to build the virtual machine where Mozilla will live. Change into the cloned git direc-
tory and run vagrant. Note you must run this command in the directory that contains the Vagrantle.
~$ cd bedrock
~bedrock-legacy/bedrock$ vagrant up
Note: The rst time you run vagrant a VM image will be downloaded and the guest machine will be
congured. You will be downloading more than 300Mb for the linux image and a bunch of additional
downloading and conguration is going to happen. The total install can take 20 minutes on a fast
machine. A decent internet connection is recommended.
Note: Often the initial installation will time out while compiling node.
If this happens just run the following command to re-sume the install:
~bedrock-legacy/bedrock$ vagrant provision
2. Update Product Details Bedrock needs to grab some information about Mozilla products to run. This is a one
time update. To run the update you need to SSH into your Vagrant install and run the update script.
SSH into your vagrant install
~bedrock-legacy/bedrock$ vagrant ssh
CD Into The Top Level Bedrock Directory:
1.2. Vagrant Installation 9
~$ cd /vagrant/
Update Product Details:
/vagrant$ python manage.py update_product_details
Exit
/vagrant$ exit
3. Conrm Everything Is Setup.
Conrm both bedrock and the legacy PHP site are working by visiting these urls. If everything looks
right you are good to go!
http://local.mozilla.org The mozilla homepage loading from bedrock.
http://local.mozilla.org/en-US/about/legal.html A legacy page loading from PHP
Note: The rst time you load a page the CSS may not load. This is likely due to the CSS not being
compiled. Doing a refresh will solve this problem.
1.2.5 Working & Workow
At this stage you should have a fully functional dev environment. You can work on les in your regular
manner and follow the normal git workow.
1.2.6 Tips & Tricks
1. Connect to your vagrant machine.
You can connect to your vagrant machine, when its running, using:
bedrock-legacy/bedrock$ vagrant ssh
2. Starting & Stopping Vagrant.
Start
~$ vagrant up
Stop (vagrant is memory intensive - so if you are not using it best to stop it):
~$ vagrant halt
1.2.7 Troubleshooting
Find us on irc in #webprod
1.3 Installing and Learning About the PHP Site
The previous version of mozilla.org was written in PHP. The PHP codebase still serves some of the mozilla.org pages
because we havent migrated everything over. A request runs through the following stack:
If the page exists in Bedrock, serve from Bedrock
If the page exists in the PHP site, serve from PHP
Else, serve a 404 page
1.3.1 History
The PHP site has a long history and as a result, is a little quirky. If you are looking to work on the site and/or set it up
locally, this page will be helpful to you.
mozilla.org, mozilla.com, and thunderbird used to be completely separate sites with different PHP codebases. In 2011
these sites were merged into one site.
The merge is purely for aesthetics though. In the PHP side of mozilla.org, a few different PHP codebases coexist
beside each other, and a combination of Apache and PHP magic bind them all together (one site to rule them all, or
something like that).
1.3.2 Installing
Apache
Whether youre installing just mozilla.com or also mozilla.org, theres some common conguration required for
Apache.
1. Install PHP. On Ubuntu, you can use these commands:
sudo apt-get install libapache2-mod-php5
2. Enable the required modules. On Ubuntu, this should get most of them:
sudo a2enmod actions expires headers php5 proxy proxy_http rewrite status vhost_alias
but if Apache fails to start with errors about unknown directives, that probably means some other module also needs
to be enabled.
Bedrock
The whole site now assumes Bedrock is also available. Even after following the instructions below, parts of the site
will not work until/unless you also have Bedrock running locally. Or you might see an old version of a page from PHP,
because the newer version is in Bedrock but the old version wasnt removed from PHP.
mozilla.com
If you want to just work on the mozilla.com codebase (currently served at mozilla.org/refox), follow these steps. You
will only get the product pages. See mozilla.org for instructions on installing the org side of the site. For more details
on why several codebases run the site, see How a Request is Handled.
Note: This assumes you are using Apache with Unix. Windows might have different steps, please contact us if you
need help.
1. Install it with these commands:
1.3. Installing and Learning About the PHP Site 11
svn co https://svn.mozilla.org/projects/mozilla.com/trunk mozilla.com
cd mozilla.com/includes
2. Open /includes/cong.inc.php and set the server_name to mozilla.local (or whatever you will use) and
le_root to the sites path on the lesystem.
3. Set up mozilla.local to resolve to localhost. This is different for each OS, but a quick way on Linux/OS X is to
add an entry to /etc/hosts:
127.0.0.1 mozilla.local
4. Congure Apache to allow the site to run with a Directory and VirtualHost directive: This
could go in the main Apache conguration le, or on Ubuntu, you might put this in
/etc/apache2/sites-available/mozilla.com.
<Directory /path/to/mozilla.com>
Options Includes FollowSymLinks MultiViews Indexes
AllowOverride All
Order Deny,Allow
Allow from all
</Directory>
<VirtualHost
*
:80>
VirtualDocumentRoot "/path/to/mozilla.com"
</VirtualHost>
Make sure to replace ServerName and /path/to/ to the correct values.
On Ubuntu, you would then enable the site with:
sudo a2ensite mozilla.com
5. You might need to set the DocumentRoot to the site if you cant load any CSS les. We are looking to x this.
DocumentRoot /path/to/mozilla/mozilla.com
You shouldnt need anything else in the site cong for mozilla.com. The .htaccess le at the root of mozilla.com
contains the rest of the required conguration.
6. Restart Apache. On Ubuntu:
sudo service apache2 restart
If you go to http://mozilla.local/ you should see a page for downloading Firefox.
mozilla.org
If you need to work on mozilla.org, you need to install it as well. The installation process is identical to mozilla.com,
with a few tweaks.
Note: htaccess les do not work on mozilla.org. If you need to add anything to htaccess les, you must commit them
to the mozilla.com codebase. See the section below about the merge for more info.
1. Make sure you install it as a subdirectory underneath mozilla.com named org.
cd mozilla.com
svn co https://svn.mozilla.org/projects/mozilla.org/trunk org
cd org/includes
2. Open /org/includes/cong.inc.php and set the server_name to mozilla.local (or whatever you will use) and
le_root to the sites path on the lesystem (including the org subdirectory).
3. In addition, set the js_prex, img_prex, style_prex cong values to /org. That is necessary.
4. If you need the archive redirects to work, you need to add the RewriteMap directives to your Apache cong for
the site. Inside the VirtualHost section that you made while installing mozilla.com, add this:
5. Depending on your system settings, you might see warnings about relying on the systems timezone settings. If
you get this, add the following to the cong.inc.php for mozilla.org:
date_default_timezone_set(America/New_York);
You can look up the correct timezone here.
That should be it. If you go to http://mozilla.local/ (or whatever local server you set it to) you should see the org home
page.
Thunderbird
The thunderbird site has been completely merged in with mozilla.org, so you can install it by installing mozilla.org. It
will be served at /thunderbird.
1.3.3 Workow
If you are working on a bug, please follow these steps:
1. Commit your work to trunk
2. Comment on the bug and add the revision in the whiteboard eld in the form r=10000. Multiple revisions
should be comma-delimited, like r=10000,10001. You can add the revision in the comment too if you want
people to have a link to the changes.
3. Add the keyword qawanted when nished
4. When all the work is done and has been QAed, mark as resolved.
We release a batch of resolved bugs every Tuesday. Other bugs can go out between releases, but by default resolved
bugs tagged with the current milestone will go out the next Tuesday.
Stage isnt used for much, but its useful for times when we are very careful about rolling out something. You typically
dont need to worry about it. When bugs are pushed live, they are pushed to stage and production at the same time.
Rolling out code
So you want to rollout a bug into production? If you look at our workow, there should be some SVN revisions logged
into the whiteboard of the bug. If not, you need to track down which revisions to push from the comments.
Once you have this list, you need to merge them to the branches tags/stage and tags/production. If the revisions are
already pushed to stage, only do the latter. These are the commands:
cd tags/stage
svn merge --ignore-ancestry -c<revs> ../../trunk
svn commit -m merged <rev> from trunk for bug <id>
<revs> is a single rev or comma-delimited like 10000,10001,10002.
Do the same for tags/production. Always format the log message like the above. You must use ignore-ancestry also
to avoid bad things.
We wrote a script to automate this if you are doing this a lot. You can nd it it on trunk in /bin/rollout. The usage
looks like this:
Usage: rollout <bug-id> <revs> <branch>
<revs> and <branch> are optional
$ cd mozilla.com # must have trunk, tags/stage, and tags/production checked out here
$ rollout 654321
Merging into tags/stage...
--- Merging r654321 into .:
<svn output>
Continue? y/n [n]y
Committing tags/stage...
Merging into tags/production...
--- Merging r654321 into .:
<svn output>
Continue? y/n [n]y
Committing tags/production...
The script parses the revisions and branch from the whiteboard data in bugzilla, and merges it from trunk to stage and
production. If the branch is already stage (b=stage in the whiteboard) it just merges it to production.
After it does the merges, it asks you if you want to continue. If you saw conicts, you shouldnt continue and you
should x the conicts and either nish the rollout by hand or update the bugzilla whiteboard and run the command
again.
1.3.4 How a Request is Handled
Magic should always be documented, so lets look at exactly how all the PHP sites work together to handle a
mozilla.org request.
mozilla.org is made up of three sites:
mozilla.com (the product pages)
mozilla.org (mofo)
mozillamessaging.com (thunderbird)
These three sites are now all merged into http://mozilla.org/. However, on the server a request can be handled by three
different codebases. Well refer to the mozilla.com codebase as moco, mozilla.org codebase as mofo, and messaging
as thunderbird.
moco is the primary codebase. A request goes through the following steps:
If the URL exists in the mofo codebase, load the page from there
If the URL exists in the thunderbird codebase, load from there
Else, let moco handle the URL like normal
The merge magic is installed into mocos htaccess and PHP les. We let moco become the primary codebase because
if theres any error in the merge code, we cant afford to break the main Firefox product pages. Theres also more
developer attention on moco.
Special Note: Only mozilla.coms .htaccess les are processed by Apache. All the others have been merged in so you
shouldnt add anything to them. Please add all htaccess rules inthe mozilla.com codebase.
Merge Magic
How we implement the merge is really important. Performance, site breakage, and amount of work to move things
around are all serious considerations. The merge is meant to be temporary as the site is moving to Python, so its not
worth the effort to literally merge all the PHP code together.
Its also important to still allow the mofo and moco codebases to be run individually. We dont want to suddenly break
it for people who have it locally checked out (short-term wise). Finally, the code of each site also dictated possible
solutions. Theres a lot of edge cases in each site so need to make sure we dont break anything.
Heres how the merge magic was implemented:
Short version:
Check out the mofo codebase under moco as the subdirectory org.
Redirect all mofo URLs to a PHP handler which loads those pages, do the same for thunderbird
Fix loading of images, css, and js by setting prex cong values and more rewrites
Merge .htaccess les into the moco codebase
Long version:
Check out the mofo codebase under moco as the subdirectory org.
Thunderbird is a folder under org, at /org/thunderbird
Generate a list of top-level folders in the org site and use Apache rewrites to redirect all those URLs to a special
php handler
Write the special php handler to load mofo pages. This is basically a port of mofos prefetch.php
Write a similar handler for the thunderbird pages and redirect all /thunderbird URLs to it
Fix loading of assets
Set cong values to load assets with the /org prex
For bad code that doesnt use the cong, use apache rewrites to redirect images and script to the respective
folder in /org. These two folders dont conict with the moco codebase. The style directory conicts, so
make sure all code uses the cong prex value.
Redirect any other asset directory to use the /org prex (/thunderbird/img/, etc)
Merge .htacess les
The biggest side effect of this is that only moco htaccess les are processed, but we should consolidate
things anyway
Move the redirects and other appropriate rules from mofos htaccess to mocos
Optimize the crazy amount of 301 and 410 redirects from mofo, mostly archive redirects, using
RewriteMap
Test to make sure everythings working, implement special rewrites or org-handler.php hacks to x any
breakage
Check le extensions for any leftover static types and rewrite them to be served by Apache
The nal result is the moco codebase which dispatches a lot of URLs to the mofo and thunderbird codebases.
1.4 Localization
The site is fully localizable. Localization les are not shipped with the code distribution, but are available on SVN:
$ git svn clone https://svn.mozilla.org/projects/mozilla.com/trunk/locales/ locale
# or
$ svn checkout https://svn.mozilla.org/projects/mozilla.com/trunk/locales/ locale
1.4.1 .lang les
Bedrock supports a workow similar to gettext. You extract all the strings from the codebase, then merge them into
each locale to get them translated.
The les containing the strings are called .lang les and end with a .lang extension.
To extract all the strings from the codebase, run:
$ ./manage.py l10n_extract
If youd only like to extract strings from certain les, you may optionally list them on the command line:
$ ./manage.py l10n_extract apps/mozorg/templates/mozorg/contribute.html
Command line glob matching will work as well if you want all of the html les in a directory for example:
$ ./manage.py l10n_extract apps/mozorg/templates/mozorg/
*
.html
That will use gettext to get all the needed localizations from python and html les, and will convert the result into
a bunch of .lang les inside locale/templates. This directory represents the reference set of strings to be
translated, and you are free to modify or split up .lang les here as needed (just make sure they are being referenced
correctly, from the code, see Which .lang le should it use?).
To merge new strings into locale directories, run:
$ ./manage.py l10n_merge
If you want to merge only specic locales, you can pass any number of them as arguments:
$ ./manage.py l10n_merge fr de
Translating with .lang les
To translate a string from a .lang le, simply use the gettext interface.
In a jinja2 template:
<div>{{ _(Hello, how are you?) }}<div>
<div>{{ _(<a href="%s">Click here</a>)|format(http://mozilla.org/) }}</div>
<div>{{ _(<a href="%(url)s">Click here</a>)|format(url=http://mozilla.org/) }}</div>
Note the usage of variable substitution in the latter examples. It is important not to hardcode URLs or other parameters
in the string. jinjas format lter lets us apply variables outsite of the string.
You can provide a one-line comment to the translators like this:
{# L10n: "like" as in "similar to", not "is fond of" #}
{{ _(Like this:) }}
The comment will be included in the .lang les above the string to be translated.
In a Python le, use lib.l10n_utils.dotlang._ or lib.l10n_utils.dotlang._lazy, like this:
from lib.l10n_utils.dotlang import _lazy as _
sometext = _(Foo about bar.)
You can provide a one-line comment to the translators like this:
# L10n: "like" as in "similar to", not "is fond of"
sometext = _(Like this:)
The comment will be included in the .lang les above the string to be translated.
Theres another way to translate content within jinja2 templates. If you need a big chunk of content translated, you
can put it all inside a trans block.
{% trans %}
<div>Hello, how are you</div>
{% endtrans %}
{% trans url=http://mozilla.org %}
<div><a href="{{ url }}">Click here</a></div>
{% endtrans %}
Note that it also allows variable substitution by passing variables into the block and using template variables to apply
them.
A general good practice is to enclose short strings in l10n calls (trans blocks or gettext wrapper). If you have a
paragraph with several sentences, it is better to wrap each sentence in its own call than the whole paragraph. That
makes it more digestable for localizers and avoids having a whole paragraph invalidated for a change to one sentence
only.
Example:
<p>
{{_(As a result, more countries and mobile phone operators will be selling Firefox in the future.)}}
{{_(Our operator partners will distribute the phones through a variety of locally-specific channels.)}}
</p>
Which .lang le should it use?
Translated strings are split across several .lang les to make it easier to manage separate projects and pages. So how
does the system know which one to use when translating a particular string?
All translations from Python les are put into main.lang. This should be a very limited set of strings and most
likely should be available to all pages.
Templates always load in main.lang, download_button.lang, and newsletter.lang
1.4. Localization 17
Additionally, each template has its own .lang le, so a template at mozorg/refox.html would use the .lang le
at <locale>/mozorg/refox.lang.
Templates can override which lang les are loaded. The above 3 global ones are always loaded, but instead
of loading <locale>/mozorg/refox.lang, the template can specify a list of additional lang les to load with a
template block:
{% add_lang_files "foo" "bar" %}
That will make the page load foo.lang and bar.lang in addition to main.lang, download_button.lang, and newslet-
ter.lang.
When strings are extracted from a template, that are added to the template-specic .lang le. If the template explicitly
species .lang les like above, it will add the strings to the rst .lang le specied, so extracted strings from the above
template would go into foo.lang.
You can similarly specify extra lang les in your Python source as well. Simply add a module-level constant in the le
named LANG_FILES. The value should be either a string, or a list of strings, similar to the add_lang_les tag above.
# forms.py
from lib.l10n_utils.dotlang import _
LANG_FILES = [foo, bar]
sometext = _(Foo about bar.)
This les strings would be extracted to foo.lang, and the lang les foo.lang, bar.lang, main.lang, down-
load_button.lang, and newsletter.lang would be searched for matches in that order.
1.4.2 l10n blocks
Bedrock also has a block-based translation system that works like the {% block %} template tag, and marks large
sections of translatable content. This should not be used very often; lang les are the preferred way to translate content.
However, there may be times when you want to control a large section of a page and customize it without caring very
much about future updates to the English page.
A Localizers guide to l10n blocks
Lets look at how we would translate an example le from English to German.
The English source template, created by a developer, lives under apps/appname/templates/appname/example.html and
looks like this:
{% extends "base.html" %}
{% block content %}
<img src="someimage.jpg">
{% l10n foo, 20110801 %}
<h1>Hello world!</h1>
{% endl10n %}
<hr>
{% l10n bar, 20110801 %}
<p>This is an example!</p>
{% endl10n %}
{% endblock %}
The l10n blocks mark content that should be localized. Realistically, the content in these blocks would be much
larger. For a short string like above, please use lang les. Well use this trivial code for our example though.
The l10n blocks are named and tagged with a date (in ISO format). The date indicates the time that this content was
updated and needs to be translated. If you are changing trivial things, you shouldnt update it. The point of l10n blocks
is that localizers completely customize the content, so they dont care about small updates. However, you may add
something important that needs to be added in the localized blocks; hence, you should update the date in that case.
When the command ./manage.py l10n_extract is run, it generates the corresponding les in the locale
folder (see below for more info on this command).
The german version of this template is created at locale/de/templates/appname/example.html. The
contents of it are:
{% extends "appname/example.html" %}
{% l10n foo %}
<h1>Hello world!</h1>
{% endl10n %}
{% l10n bar %}
{% endl10n %}
This le is an actual template for the site. It extends the main template and contains a list of l10n blocks which override
the content on the page.
The localizer just needs to translate the content in the l10n blocks.
When the reference template is updated with new content and the date is updated on an l10n block, the generated l10n
le will simply add the new content. It will look like this:
{% extends "appname/example.html" %}
{% l10n foo %}
<h1>This is an English string that needs translating.</h1>
{% was %}
<h1>Dies ist ein English string wurde nicht.</h1>
{% endl10n %}
{% l10n bar %}
{% endl10n %}
Note the was block in foo. The old translated content is in there, and the new content is above it. The was content
is always shown on the site, so the old translation still shows up. The localizer needs to update the translated content
and remove the was block.
Generating the locale les
$ ./manage.py l10n_check
This command will check which blocks need to be translated and update the locale templates with needed translations.
It will copy the English blocks into the locale les if a translation is needed.
You can specify a list of locales to update:
1.4. Localization 19
$ ./manage.py l10n_check fr
$ ./manage.py l10n_check fr de es
1.5 Developing on Bedrock
1.5.1 Writing URL Patterns
URL patterns should be as strict as possible. It should begin with a ^ and end with /$ to make sure it only matches what
you speciy. It also forces a trailing slash. You should also give the URL a name so that other pages can reference it
instead of hardcoding the URL. Example:
url(r^channel/$, channel, name=mozorg.channel)
Bedrock comes with a handy shortcut to automate all of this:
from bedrock.mozorg.util import page
page(channel, mozorg/channel.html)
You dont even need to create a view. It will serve up the specied template at the given URL (the rst parameter).
You can also pass template data as keyword arguments:
page(channel, mozorg/channel.html, latest_version=product_details.refox_versions[LATEST_FIREFOX_VERSION])
The variable latest_version will be available in the template.
1.5.2 Embedding images
Images should be included on pages using helper functions.
media()
For a simple image, the media() function is used to generate the image URL. For example:
<img src="{{ media(img/firefox/new/firefox-logo.png) }}" alt="Firefox" />
will output an image:
<img src="/media/img/firefox/new/firefox-logo.png" alt="Firefox">
high_res_img()
For images that include a high-resolution alternative for displays with a high pixel density, use the high_res_img()
function:
high_res_img(img/firefox/new/firefox-logo.png, {alt: Firefox, width: 200, height: 100})
The high_res_img() function will automatically look for the image in the URL parameter sufxed with -high-res,
e.g. img/refox/new/refox-logo-high-res.png and switch to it if the display has high pixel density.
high_res_img() supports localized images by setting the l10n parameter to True:
high_res_img(img/firefox/new/firefox-logo.png, {l10n: True, alt: Firefox, width: 200, height: 100})
When using localization, high_res_img() will look for images in the appropriate locale folder. In the above example,
for the de locale, both standard and high-res versions of the image should be located at media/img/l10n/de/refox/new/.
l10n_img()
Images that have translatable text can be handled with l10n_img():
<img src="{{ l10n_img(firefox/os/have-it-all/messages.jpg) }}" />
The images referenced by l10n_img() must exist in media/img/l10n/, so for above example, the images
could include media/img/l10n/en-US/refox/os/have-it-all/messages.jpg and media/img/l10n/es-ES/refox/os/have-it-
all/messages.jpg.
platform_img()
Finally, for outputting an image that differs depending on the platform being used, the platform_img() function will
automatically display the image for the users browser:
platform_img(img/firefox/new/browser.png, {alt: Firefox screenshot})
platform_img() will automatically look for the images browser-mac.png, browser-win.png, browser-linux.png, etc.
Platform image also supports hi-res images by adding data-high-res: true to the list of optional attributes.
platform_img() supports localized images by setting the l10n parameter to True:
platform_img(img/firefox/new/firefox-logo.png, {l10n: True, alt: Firefox screenshot})
When using localization, platform_img() will look for images in the appropriate locale folder. In the above example,
for the es-ES locale, all platform versions of the image should be located at media/img/l10n/es-ES/refox/new/.
1.5.3 Writing Views
You should rarely need to write a view for mozilla.org. Most pages are static and you should use the page expression
documented above.
If you need to write a view and the page has a newsletter signup form in the footer (most do), make sure to handle this
in your view. Bedrock comes with a function for doing this automatically:
from bedrock.mozorg.util import handle_newsletter
from django.views.decorators.csrf import csrf_exempt
@csrf_exempt
def view(request):
ctx = handle_newsletter(request)
return l10n_utils.render(request, app/template.html, ctx)
Youll notice a few other things in there. You should use the l10n_utils.render function to render templates because
it handles special l10n work for us. Since were handling the newsletter form post, you also need the csrf_exempt
decorator.
Make sure to namespace your templates by putting them in a directory named after your app, so instead of tem-
plates/template.html they would be in templates/blog/template.html if blog was the name of your app.
1.5.4 Python and Django Style
See the Mozilla Coding Standards.
1.5. Developing on Bedrock 21
1.6 How to contribute
1.6.1 Git workow
When you want to start contributing, you should create a branch from master. This allows you to work on different
project at the same time:
git checkout master
git checkout -b topic-branch
To keep your branch up-to-date, assuming the mozilla repository is the remote called mozilla:
git fetch mozilla
git checkout master
git merge mozilla/master
git checkout topic-branch
git rebase master
If you need more Git expertise, a good resource is the Git book.
Once youre done with your changes, youll need to describe those changes in the commit message.
1.6.2 Git commit messages
Commit messages are important when you need to understand why something was done.
First, learn how to write good git commit messages.
All commit messages must include a bug number. You can put the bug number on any line, not only the rst
one.
If you use the syntax bug xxx, Github will reference the commit into Bugzilla. With fix bug xxx, it will
even close the bug once it goes into master.
If youre asked to change your commit message, you can use these commands:
git commit --amend
# -f is doing a force push because you modified the history
git push -f my-remote topic-branch
1.6.3 Submitting your work
In general, you should submit your work with a pull request to master. If you are working with other people or you
want to put your work on a demo server, then you should be working on a common topic branch.
Once your code has been positively reviewed, it will be deployed shortly after. So if you want feedback on your code
but its not ready to be deployed, you should note it in the pull request.
1.6.4 Getting a new Bedrock page online
On our servers, Bedrock pages are accessible behind the /b/ prex. So if a page is accessible at this URL locally:
http://localhost:8000/foo/bar
then on our servers, it will be accessible at:
http://www.mozilla.org/b/foo/bar
When youre ready to make a page available to everyone, we need to remove that /b/ prex. We handle that with
Apache RewriteRule. Apache cong les that are included into the servers cong are in the bedrock code base in
the etc/httpd directory. In there youll nd a le for each of the environments. Youll almost always want to
use global.conf unless you have a great reason for only wanting the cong to stay on one of the non-production
environments.
In that le youll add a RewriteRule that looks like the following:
# bug 123456
RewriteRule ^/(\w{2,3}(?:-\w{2}(?:-mac)?)?/)?foo/bar(/?)$ /b/$1foo/bar$2 [PT]
This is a lot simpler than it looks. The rst large capture is just whats necessary to catch every possible locale code.
After that its just your new path. Always capture the trailing slash as we want that to hit django so it will redirect.
Note: Its best if the RewriteRule required for a new page is in the original pull request. This allows it to ow through
the push process with the code and for it to go live as soon as its on the production server. Its also one less review
and pull-request for us to manage.
1.6.5 Server architecture
Demos
URLs: http://www-demo1.allizom.org/ , http://www-demo2.allizom.org/ and http://www-demo3.allizom.org/
PHP SVN branch: trunk, updated every 10 minutes
Bedrock locale SVN branch: trunk, updated every 10 minutes
Bedrock Git branch: any branch we want, manually updated
Dev
URL: http://www-dev.allizom.org/
PHP SVN branch: trunk, updated every 10 minutes
Bedrock Git branch: master, updated every 10 minutes
Stage
URL: http://www.allizom.org/
PHP SVN branch: tags/stage, updated every 10 minutes
Bedrock Git branch: master, updated manually
Production
URL: http://www.mozilla.org/
PHP SVN branch: tags/production, updated every 10 minutes
Bedrock Git branch: master, updated manually
1.6. How to contribute 23
We use Chief for the manual deploys. You can check the currently deployed git commit by checking
https://www.mozilla.org/media/revision.txt.
If you want to know more and you have an LDAP account, you can check the IT documentation.
1.6.6 Pushing to production
Were doing pushes as soon as new work is ready to go out.
After doing a push, the pusher needs to update the bugs that have been pushed with a quick message stating that the
code was deployed. Chief will send on #www a URL with all commits that have been deployed.
If youd like to see the commits that will be deployed before the push run the following command:
./bin/open-compare.py
This will discover the currently deployed git hash, and open a compare URL at github to the latest master. Look at
open-compare.py -h for more options.
1.7 Newsletters
Bedrock includes support for signing up for and managing subscriptions and preferences for Mozilla newsletters.
By default, every pages footer has a form to signup for the default newsletter, Firefox & You.
1.7.1 Features
ability to subscribe to a newsletter from a pages footer area. Many pages on the site might include this.
whole pages devoted to subscribing to one newsletter, often with custom text, branding, and layout
newsletter preference center - allow user to change their email address, preferences (e.g. language, HTML vs.
text), which newsletters theyre subscribed to, etc. Access is limited by requiring a user-specic token in the
URL (its a UUID). The full URL is included as a link in each newsletter sent to the user, which is the only way
(currently) they can get the token.
landing pages that user ends up on after subscribing. These can vary depending on where theyre coming from.
1.7.2 Newsletters
Newsletters have a variety of characteristics. Some of these are implemented in Bedrock, others are transparent to
Bedrock but implemented in the basket back-end that provides our interface to the newsletter vendor.
Public name - the name that is displayed to users, e.g. Firefox Weekly Tips.
Internal name- a short string that is used internal to Bedrock and basket to identify a newsletter. Typically these
are lowercase strings of words joined by hyphens, e.g. refox-tips. This is what we send to basket to identify
a newsletter, e.g. to subscribe a user to it.
Show publicly - pages like the newsletter preferences center show a list of unsubscribed newsletters and allow
subscribing to them. Some newsletters arent included in that list by default (though they are shown if the user
is already subscribed, to let them unsubscribe).
Languages - newsletters are available in a particular set of languages. Typically when subscribing to a newsletter,
a user can choose their preferred language. We should try not to let them subscribe to a newsletter in a language
that it doesnt support.
The backend only stores one language for the user though, so whenever the user submits one of our forms,
whatever language they last submitted is what is saved for their preference for everything.
Welcome message - each newsletter can have a canned welcome message that is sent to a user when they
subscribe to it. Newsletters should have both an HTML and a text version of this.
Drip campaigns - some newsletters implement so-called drip campaigns, in which a series of canned messages
are dribbled out to the user over a period of time. E.g. 1 week after subscribing, they might get message 1; a
week later, message 2, and so on until all the canned messages have been sent.
Because drip campaigns depend on the signup date of the user, were careful not to accidentally change the
signup date, which could happen if we sent redundant subscription commands to our backend.
1.7.3 Bedrock and Basket
Bedrock is the user-facing web application. It presents an interface for users to subscribe and manage their subscrip-
tions and preferences. It does not store any information. It gets all newsletter and user-related information, and makes
updates, via web requests to the Basket server.
The Basket server implements an HTTP API for the newsletters. The front-end (Bedrock) can make calls to it to
retrieve or change users preferences and subscriptions, and information about the available newsletters. Basket im-
plements some of that itself, and other functions by calling the newsletter vendors API. Details of that are outside the
scope of this document, but its worth mentioning that both the user token (UUID) and the newsletter internal name
mentioned above are used only between Bedrock and Basket.
1.7.4 URLs
Here are a few important URLs implemented. These were established before Bedrock came along and so are unlikely
to be changed.
(Not all of these might be implemented in Bedrock yet.)
/newsletter/ - subscribe to mozilla-and-you newsletter (public name: Firefox & You)
/newsletter/hacks.mozilla.org/ - subscribe to app-dev newsletter (Firefox Apps & Hacks). This one is displayed as
a frame inside some other page(s), so it works differently than the other signup pages.
/newsletter/existing/USERTOKEN/ - user management of their preferences and subscriptions
1.7.5 Conguration
Currently, information about the available newsletters is congured in Basket. See Basket for more information.
1.7.6 Footer signup
Customize the footer signup form by overriding the email_form template block. For example, to have no signup form:
{% block email_form %}{% endblock %}
The default is:
1.7. Newsletters 25
{% block email_form %}{{ email_newsletter_form() }}{% endblock %}
which gives a signup for Firefox & You. You can pass parameters to the macro email_newsletter_form to
change that. For example, the newsletter_id parameter controls which newsletter is signed up for, and title
can override the text:
{% block email_form %}
{{ email_newsletter_form(app-dev,
_(Sign up for more news about the Firefox Marketplace.)) }})
{% endblock %}
Pages can control whether country or language elds are included by passing include_language=[True|False] and/or
include_country=[True|False].
You can also use the same form outside a page footer by passing footer=False to the macro.
1.7.7 Creating a signup page
Start with a template that extends newsletter/one_newsletter_signup.html. Its probably simplest
to copy an existing one, like newsletter/mobile.html.
Set the newsletter_title and newsletter_id variables and override at least the page_title and
newsletter_content blocks:
Then add a url to newsletter/urls.py:
# "about:mobile"
page(newsletter/about_mobile, newsletter/mobile.html),
1.8 Tabzilla
Tabzilla is the universal tab displayed on Mozilla websites.
Adding the universal tab to a site requires:
1. Add the static tab link (example below) to the top of your template:
<a href="https://www.mozilla.org/" id="tabzilla">mozilla</a>
2. Include the tabzilla.css CSS le either as a CSS include or built in to your minied styles:
<link href="//mozorg.cdn.mozilla.net/media/css/tabzilla-min.css" rel="stylesheet" />
3. Include the tabzilla.js le in your template (preferably in the footer):
<script src="//mozorg.cdn.mozilla.net/tabzilla/tabzilla.js"></script>
This will choose the best locale for your visitor. If you prefer to force the locale, you can use:
<script src="//mozorg.cdn.mozilla.net/{locale}/tabzilla/tabzilla.js"></script>
Where {locale} is the language in which youd like Tabzilla to be loaded (e.g. fr or de). If Tabzilla is not yet
translated into said locale the user will get the en-US version.
Note: Tabzilla uses jQuery. If your site already includes jQuery be sure to place the Tabzilla script tag after the one
for jQuery. Tabzilla will use the existing jQuery if available and a supported version, otherwise it will load its own
version of jQuery.
That the source le URLs begin with // is not a typo. This is a protocol-relative URL which allows the resource to
be loaded via whichever protocol (http or https) the page itself is loaded. This removes the need to add any logic to
support loading Tabzilla over both secure and insecure connections, thereby avoiding mixed-content warnings from
the browser.
1.8.1 Requirements
As the universal tab does inject HTML/CSS into the DOM, some there are some requirements that you must meet.
Background images must not be attached to the <body> element.
Absolutely positioned elements must not be positioned relative to the <body> element.
An element other than the <body> should add a 2 pixel white border to the top of the page (border-top:
2px solid #fff;)
Any background image or absolutely positioned element attached to the body element would not move with the rest
of the contents when the tab slides open. Instead, any such background or element should be attached to anoter HTML
element in the page (a wrapper div, for example). Note that this issue does not apply to solid background colors, or
backgrounds that do not vary vertically (solid vertical stripes, for example).
If jQuery is already included on the page, it will be used by Tabzilla. If jQuery is not already on the page, it will
automatically be included after the page has loaded.
1.8.2 Translation Bar
Tabzilla has an opt-in extension called Translation Bar that automatically offers a link to a localized page, if available,
based on the users locale. It is intended to improve international user experience.
Adding the Translation Bar extension to Tabzilla requires:
1. Include alternate URLs in the <head> element. For example:
<link rel="alternate" hreflang="en-US" href="http://www.mozilla.org/en-US/firefox/new/" title="English (US)">
<link rel="alternate" hreflang="fr" href="http://www.mozilla.org/fr/firefox/new/" title="Franais">
The Translation Bar alternatively detects available translations by looking for a language switcher like below,
but implementation of alternate URLs is recommended also from the SEO perspective:
<select id="language"><option value="en-US">English (US)</option></select>
2. Add the data-infobar attribute to the tab link, with the translation option:
<a href="https://www.mozilla.org/" id="tabzilla" data-infobar="translation">mozilla</a>
Note: Though the Translation Bar is currently implemented as an extension of Tabzilla, it might be moved to a
standalone language utility in the future.
1.8. Tabzilla 27

Bedrock

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Bedrock

Caricato da

Copyright:

Formati disponibili

playdoh Documentation

Potrebbero piacerti anche