1.5-M1 TO 1.5-M2 UPGRADE

Daisy 1.5 Milestone 2 was released on June 21, 2006.

Changes

Important changes: read this section!

  • Daisy Wiki: the configuration and data used by the Daisy Wiki is now stored in a separate wikidata directory, similar to the data directory of the repository server. The data directory contains things such as the skins, the site definitions, the doctype XSLs, the logs, ... In earlier Daisy versions, these items were simply stored inside the Daisy Wiki webapp. The new structure is also illustrated in the updated deployment diagram.
    The introduction of the wikidata directory has several advantages:
    • Clear separation of data and application
    • Clear view on official extension points
    • Upgrading and backups become easier: it will no longer be needed to copy over your site definitions, skins, etc. after upgrading.
  • OpenJMS is replaced with ActiveMQ as default JMS implementation.
    • We embedded the ActiveMQ inside the repository server so from now on, the OpenJMS server doesn't need to be started anymore.
    • The ActiveMQ project is much more alive then OpenJMS, provides more ways to connect to it, ...
  • A new default skin
    • Much more CSS-based, the layout is better controllable through CSS only.
    • The rather big default.css file has been split up in multiple smaller CSS files
    • Aesthetic improvements:
      • lighter look
      • fixed width layout (better readability on wide screens)
      • ...
    • See also the compatibility notes if you have created your own skin(s) based on the old default skin.

Other new features

  • Allow to switch between live and staging views in the Wiki. Up till now, the Wiki displayed by default the live versions of all documents. When switching to staging mode, the last versions of the documents are displayed by default. So it shows how everything would look if the last versions of all documents would become live. This also applies to navigation trees, queries embedded in documents, et cetera. See docs.
  • Navigation trees:
    • Nodes can now be hidden or visible-when-active. Hidden nodes allow to have the navigation tree expanded up to a certain point, and to control the URL path, without having to overload the navigation tree (this will probably mostly be done using queries). Visible-when-active is similar: the node is always hidden except when it is active.
    • Query nodes in the navigation tree can now create hierarchies instead of just a flat list of nodes. For example, if you enter a query like:
      select documentType, name where true order by documentType, name

      this will create group nodes per document type, and normal document nodes below that. This can of course work multiple levels deep. Make sure the order by clause is meaningful (thus: sort values in the same order as selected).

  • Books:
    • A new book publication process task, named "custom", allows to specify a Java class to do custom processing.
    • Publication types can now specify parts for which they need the binary content to be available.
    • The input for the doctype XSLs now has a /document/@isIncluded attribute, just as in the Wiki, to distinguish included documents.
    • Added two publication tasks: renderSVG and callPipeline, both of which are useful in combination with the new part-download capability (see the second bullet)
    • The publication task copyBookInstanceImages has been renamed to copyBookInstanceResources, though the old name still works. The new name also comes with some new functionality: not only img/@src resources are copied, but also resources linked to in a/@href, embed/@src, object/@data.
  • Faceted navigation: allow to limit the set of documents on which the faceted navigation is performed, through an <defaultConditions> element in the faceted navigation definition.
  • Removed the need to have absolute paths in the myconfig.xml by supporting property substitutions (where needed), such as ${daisy.datadir} and ${daisy.home}. The repository initialisation script now generates such myconfig.xml's without absolute paths. This allows to move the data directory around without having to adjust these paths. (DSY-271)
  • The homepage of a site, as defined in the siteconf.xml, can now be a custom path instead of only a document ID. This is for example useful to let the homepage point to some extension URL. See the sites documentation.
  • The recent changes page now has links to the RSS feeds (including link elements in the page header, so that the feeds can be detected automatically).
  • Editor: improvements to the image browser dialog:
    • the collection of the current site is automatically selected by default, so the user immediately sees the images in that collection. The list of collections has been changed to a drop-down list.
    • it is possible to select no collection, thus showing all images, without a collection constraint
    • the image browser now shows all documents that have an 'ImageData' part. Previously it showed all documents of type 'Image'. The benefit of this is that you can have custom document types for images, and as long as they reuse the 'ImageData' part for the rendered image, it will now show up in the image browser.
  • Document editor:
    • Allow to initialise new documents with some content before showing the editor (docs).
    • Allow to specify a custom location to redirect to after editing is done (useful for included/aggregated content).
  • Included documents now show edit links next to their title(if you have edit rights on the document) (similar to Wikipedia).
  • The document basket: a new Daisy Wiki feature that allow end-users to do things with sets of documents (such as aggregate them for easy printing), as described in this mail.
  • Added a schema uploader tool, which allows to create a Daisy repository schema (document, part, field types) from an XML file.
  • Added SVG support, through a new SvgDocument document type. This is roughly based on the scratchpad article, but with several enhancements: the SVGs embedded in HTML are automatically resized by some javascript, and the SVGs are also properly rendered in the books (in the PDF publication by default at 250dpi).
  • The document type specific XSLs, and the query styling XSLs are now put in the directory of the skin. This makes fallback between skins work for them.
  • Added a version reversion feature in the Wiki, accessible through the versions overview page.

Bug fixes & minor improvements

  • Some new bugs in 1.5-M1 were fixed: image upload dialog not working, create document task not working
  • The ACL admin screen now uses icons to indicate the ACL actions (grant, deny, do nothing), which makes the layout fit better on the screen and is easier to read.
  • Fixed a problem with the backup restore on Windows: the tool didn't wait for the external processes (mysql/mysqladmin) it calls to end.
  • Added a patched merlin.bat file, so that when you have Java installed in a directory containing spaces, you don't need to patch this manually anymore.
  • In Internet Explorer, links consisting of only a fragment identifiers (#something) were prepended with the filename from the current (editor) URL.
  • We build Cocoon now with the latest Rhino version (1.6-R2), giving access to its latest bug fixes and features (such as E4X). Also, this was required to get javascripts embedded in SVGs working.
  • Books: the default name proposed for book instances contained the colon character (:), which is a prohibited file name character on Windows.
  • Editor: when switching between td and th cells, the colspan and rowspan were not preserved.
  • And much more small polishing...

Compatibility notes

Skins

[todo: add helpful notes here on updating custom skins based on the old default skin]

Custom book publication types

If you made custom book publication types or document type specific XSLs for use in books, they might need some small changes due to the introduction of the wikidata directory.

In the query-styling.xsl of the publication type, if you want to import the default query styling XSL, the include should now be:

<xsl:include href="wikidata:books/publicationtypes/common/default-query-styling.xsl"/>

If in the custom document type specific XSL you included the default book-document-to-html.xsl file, the import should now be:

<xsl:include href="wikidata:books/publicationtypes/common/book-document-to-html.xsl"/>

Databases

MySQL 4.0 is not supported anymore. You might still be able to get it to work, but it wasn't worth the effort anymore for us to support it in the default configuration.

JmsClient API

If you were using the remote Java API and constructing a JmsClientImpl to enable remote cache invalidation through JMS, you'll notice the constructor has changed: it takes a new parameter clientID (the JMS client ID, which is a string of your choice, but should be different for each client), and the topicConnectionFactory and queueConnectionFactory parameters have merged into just one connectionFactory argument (since we're now using JMS 1.1 APIs internally, as a result of the switch to ActiveMQ).

Upgrading

Important: these are instructions for upgrading from Daisy 1.5-M1. To upgrade from older releases, e.g. Daisy 1.4, please follow the aggregated instructions for upgrading from 1.4 to 1.5-final, which can be found here.

Download and extract installation files

  1. Rename your existing <DAISY_HOME> directory. During these instructions, we will refer to this location as <OLD_DAISY_HOME>.
  2. Download the Daisy 1.5 installation files from here.
  3. Extract the installation archive. Make sure that  your <DAISY_HOME> variable points to that newly created directory, if necessary, move or rename that directory (or adapt your <DAISY_HOME> environment variable).

Switching from OpenJMS to ActiveMQ

Make sure no messages are left in OpenJMS

Before moving to ActiveMQ, it is a good idea to check if there are no messages waiting to be processed in OpenJMS's queues. Usually JMS messages in Daisy are processed quickly, so unless you have had high activity right before shutting down the Daisy repository, there will normally be no more messages waiting to be processed.

To be sure, you can do a quick check like this:

    1. start OpenJMS
    2. go to OPENJMS_HOME/bin, and execute the "admin" script. This will launch a GUI window. In its menu, choose Actions, Connections, Online. It will ask for a username and password. The username is normally "admin", the password can be found in the file OPENJMS_HOME/config/openjms.xml (look for <User name="admin" password="something"/>). Once connected, you see the topics with their durable subscribers, and the queues. Next to them a number is displayed, they should all be zero.

If there would still be messages, just start the repostory server again, and wait till they have all been processed.

Creating a database for ActiveMQ

Start the MySQL client:

mysql -uroot

and execute:

CREATE DATABASE activemq;
GRANT ALL ON activemq.* TO activemq@'%' IDENTIFIED BY 'activemq';
GRANT ALL ON activemq.* TO activemq@localhost IDENTIFIED BY 'activemq';

ActiveMQ will automatically create its database tables the first time the repository server is launched.

ActiveMQ configuration

Create the ActiveMQ configuration :

cp <DAISY_HOME>/repository-server/conf/activemq-conf.xml.template <DAISY_DATA>/conf/activemq-conf.xml
cp <DAISY_HOME>/repository-server/conf/login.config <DAISY_DATA>/conf/
cp <DAISY_HOME>/repository-server/conf/users.properties <DAISY_DATA>/conf/
cp <DAISY_HOME>/repository-server/conf/groups.properties <DAISY_DATA>/conf/

Update myconfig.xml (<DAISY_DATA>/conf/myconfig.xml):

Look for the following element:

<target path="/daisy/jmsclient/jmsclient">

and replace the <configuration> element inside it with:

<configuration>
  <jmsConnection>
    <clientId>daisy-repository</clientId>
    <credentials username="admin" password="jmsadmin"/>
    <initialContext>
      <property name="java.naming.provider.url" value="vm://DaisyJMS?brokerConfig=xbean:file:${daisy.datadir}/conf/activemq-conf.xml"/>
      <property name="java.naming.factory.initial" value="org.apache.activemq.jndi.ActiveMQInitialContextFactory"/>
      <property name="queue.fullTextIndexerJobs" value="fullTextIndexerJobs"/>
      <property name="topic.daisy" value="daisy"/>
    </initialContext>
    <connectionFactoryName>ConnectionFactory</connectionFactoryName>
  </jmsConnection>
</configuration>

If you are using MySQL 5 (thus not 4.1), then you need to edit <DAISY_DATA>/conf/activemq-conf.xml. Look for the following line (here split on two lines for readability):

<property name="url" value="jdbc:mysql://localhost/activemq?relaxAutoCommit=true&amp;
                         useServerPrepStmts=false&amp;sessionVariables=storage_engine=InnoDB"/>

In this, you should remove the "useServerPrepStmts=false&amp;", so that it becomes:

<property name="url" value="jdbc:mysql://localhost/activemq
          ?relaxAutoCommit=true&amp;sessionVariables=storage_engine=InnoDB"/>

Configuring the Driver Registrar component

This new component registers JDBC drivers that are used by datasources in the repository-server.

Background info: previously, the JDBC drivers were configured as part of the datasource, but since JDBC connections are now needed by multiple datasources in the same VM (both the repository server and ActiveMQ), their registration is now moved into a specific component.

Edit <DAISY_DATA>/conf/myconfig.xml, and add a new target :

<target path="/daisy/driverregistrar/driverregistrar">
  <configuration>
    <drivers>
      <driver>
        <classpath>${daisy.home}/lib/mysql/jars/mysql-connector-java-3.1.12-bin.jar</classpath>
      	<classname>com.mysql.jdbc.Driver</classname>
      </driver>
    </drivers>
  </configuration>
</target>

Notice the upgrade from mysql-connector 3.1.7 to mysql-connector 3.1.12

You may now also remove the references to JDBC drives from the datasource component.  In the same file look for "/daisy/datasource/datasource".  In the configuration of this component look for the "driverClasspath" and "driverClassName" elements.  You may simply remove both these elements like this :

<target path="/daisy/datasource/datasource">
    <configuration>
      <username>daisy</username>
      <password>daisy</password>
      <url>jdbc:mysql://localhost/daisyrepository?useServerPrepStmts=false</url>
      <driverClasspath>REMOVE THIS LINE</driverClasspath>
      <driverClassName>REMOVE THIS LINE</driverClassName>
      <maxActive>20</maxActive>
      <maxIdle>8</maxIdle>
      <minIdle>0</minIdle>
      <maxWait>5000</maxWait>
    </configuration>
  </target>

Updating wrapper configuration for the Repository Server

If you are using the service wrapper (or custom startup scripts) to start the repository server, then you will need to upgrade the configuration/scripts to pass some additional parameters. In dsy_repo_wrapper.conf, these have to be added (see also the full script):

wrapper.java.additional.5=-Dorg.apache.commons.logging.Log=org.outerj.daisy.logging.DaisyLog
wrapper.java.additional.6=-Djava.security.auth.login.config=%DAISY_DATADIR%/conf/login.config
wrapper.java.additional.7=-Ddaisy.datadir=%DAISY_DATADIR%
wrapper.java.additional.8=-Ddaisy.home=%DAISY_HOME%

Optional: remove absolute paths in myconfig.xml

Daisy now allows to use properties in the myconfig.xml to avoid absolute paths. If you want, you can update your myconfig.xml and change absolute references to the daisy data directory and daisy home directory by ${daisy.datadir} and ${daisy.home} respectively. Doing this will allow to move your data directory around without changing the myconfig.xml.

Start repository server

At this point, start the repository server, as this is needed for the remainder of the upgrade.

It is not needed anymore to start OpenJMS.

Create wikidata directory

From now on, configurations parameters, skins, doctype XSLs, the book publication types and book store, the wiki log files, in short all 'wiki-instance-specific' stuff is stored in a separate "wikidata directory" (not to be confused with the daisydata directory, which performs a similar role but then for the repository server).

We will now create such a directory and copy your existing configuration to it.

To create the wikidata directory, go to <DAISY_HOME>/install and execute:

daisy-wikidata-init

The program will ask you for a location for this directory. Just choose some location for this, for example put it next to (but not inside) the repository daisydata directory.

The program will also ask for the location of the repository daisydata directory, as it will take some configuration values from there (internal & jms user password) and put them in the wiki config.

It will also try to create the registrar user, but since you're doing an upgrade that user will already exist so it will simply skip that. However, you will need to enter the password for the registrar user. This can be found in <OLD_DAISY_HOME>/daisywiki/webapp/WEB-INF/cocoon.xconf, and look there for something like:

<registrarUser login="registrar" password="b6f84c35249a08d11ad6fa8279e2f9"/>

Just copy-and-paste that password to the input prompt.

Once the daisy-wikidata-init script has ended, we can copy over the remaining things:

  • If modified, copy <OLD_DAISY_HOME>/daisywiki/webapp/daisy/external-include-rules.xml to <wikidata dir>/external-include-rules.xml
  • Copy <OLD_DAISY_HOME>/daisywiki/webapp/daisy/sites/* (without the cocoon subdir) to <wikidata dir>/sites
  • If you created your own skins, copy <OLD_DAISY_HOME>/daisywiki/webapp/daisy/resources/skins/(yourskin) to <wikidata dir>/resources/skins
  • If you created your own doctype XSLs, then these also have to be copied into the wikidata directory, but at a new location, namely inside the directory of the skin. For example:
    <OLD_DAISY_HOME>/daisywiki/webapp/daisy/resources/document-styling/default/html/MyDoctype.xsl to <wikidata dir>/resources/skins/default/document-styling/html/MyDoctype.xsl
  • Ditto for query-styling XSLs: copy all contents from <OLD_DAISY_HOME>/daisywiki/webapp/daisy/resources/ to <wikidata dir>/resources/skins/<skin-name>/query-styling/
  • If you published any books, copy the from <OLD_DAISY_HOME>/daisywiki/webapp/daisy/bookstore to <wikidata dir>/bookstore
  • If you made any custom book publication types, you can copy these from <OLD_DAISY_HOME>/daisywiki/webapp/daisy/books/publicationtypes to <wikidata dir>/books/publicationtypes

If you made any configuration changes to the repository manager, JMS client or registrar component which were previously in the cocoon.xconf, then you can now find these in <wikidata dir>/daisy.xconf.

Updating wrapper configuration for the Daisy Wiki

Similarly as in the previous section, the Daisy Wiki needs the following additional parameter in dsy_wiki_wrapper.conf (see also the full script):

wrapper.java.additional.7=-Ddaisywiki.data=%DAISYWIKI_DATADIR%

The DAISYWIKI_DATADIR environment variable has to be declared in the shell script (dsy_wiki.sh or equivalent):

#! /bin/sh

DAISY_HOME=/home/daisy/daisy ; export DAISY_HOME
JETTY_HOME=$DAISY_HOME/daisywiki/jetty ; export JETTY_HOME
DAISYWIKI_DATADIR=/path/to/your/daisywikidata ; export DAISYWIKI_DATADIR

[.....]

Start the Daisy Wiki

Now you can also start the Daisy Wiki.

After-upgrade notes

OpenJMS cleanup

Since OpenJMS is not used anymore, some cleanup you might want to to:

  • delete the openjms database
  • if globally set, remove the OPENJMS_HOME environment variable
  • remove the openjms wrapper script, if used

ActiveMQ authentication

With the procedure we followed above, ActiveMQ still uses default passwords. If the ActiveMQ port is publicly accessible, it is of course very much recommended to change these. Here is how. The password needs to be changed in the following locations:

Original setting:

In  this file:
<datadir>/conf/users.properties

in this line:
admin=jmsadmin

For the connection made by the Repository Server:

In this file:
<datadir>/conf/myconfig.xml

in this line:
<credentials username="admin" password="jmsadmin"/>

For the connection made by the Daisy Wiki:

In this file:
<wikidata dir>/daisy.xconf

in this line:
<credentials username="admin" password="jmsadmin"/>

After making these changes, stop both the Daisy Wiki and the Repository Server, and then restart them (first the repository, then the wiki).