Calibre2Opds Troubleshooting

Home Page	Calibre2opds User Guide						Developer Guide
	Overview	Installing	Generating	Publishing	Using	Troubleshooting

Troubleshooting Calibre2opds

We have tried to make calibre2opds as easy to use as we can. However, inevitably, some users will encounter problems when using calibre2opds either because we have not made things as clear as we thought, or because there are bugs in the software. This section is intended to give guidance on how to proceed if you encounter problems in using Calibre2Opds. It will help you identify the cause of your specific problem and give fixes for problems that other users have reported encountering.

The topic areas covered in this section are:

Common Problems

This section is a selection of particular issues that users have found, and the steps that can be taken to both diagnose and fix these issues. It will be progressively extended as more items come to light.

• I get an error on Windows trying to run calibre2opds indicating javaw.exe cannot be found.
  This indicates that the Calibre2opds startup file (runguicmd in GUI mode, run.cmd in batch mode) did not manage to locate the JAVA binaries. The Java install did not add the Java binaries to your search path or set up the JAVA_HOME environment variable to say where Java has been installed and the startup file was unable to automatically determine suitable values for these. In such a case the Java binaries folder needs to be added to the PATH environment variable, or the JAVA_HOME environment variable needs to be set up. If using Windows 7/Windows Vista/Windows 8, the easiest way to do this is as follows:
  
  • Go to the Start Menu and right-click Computer and select Properties from the menu displayed</li>
  • Select the Advanced System Settings option
  • Click the Environment Variables button
  • In the System Variables area, scroll down until you can see the JAVA_HOME environment variable. Click on this to highlight it and press the Edit button if it is shown, or press the New button if it is not shown.
  • if you had to press the New button then give the name JAVA_HOME to the environment variable.
  • add the Java binaries install locationin the value field. It will vary according to where you installed Java, but is typically something like 'C:\Program Files\Java\jre6.Press OK to accept the changes. You may have to press OK on several times to close down all the dialog.You can check your changes by starting a new Command Prompt window (it must not be an existing Command Prompt window as you need a new one to pick up the change you have just made). Typing
    SET PATH
    which will display the current setting of the PATH environment variable with all variables expanded. The path to the Java binaries should now be in the list if you have made the change correctly.
    
    

• Heap Error or Stack Error during a generate run
  It is possible that the Java VM that is running Calibre2opds may give an error message that indicates it has run out of either stack space or heap space during a generate run. You can increase (or decrease) the space allocated to the java VM by editing the files rungui.cmd/rungui.sh (GUI mode) or run.cmd/run.sh (Batch mode) that are used to start Calibre2opds. These are text files that can be edited with any standard text editor (e.g. Notepad on Windows). You will see some entries at the top of these files (which are liberally commented) where the values that are going to be used are set.
  
  The values that are preset on installation are ones that are appropriate for the vast majority of systems. If you are using a system that is very short on RAM then you may want to consider decreasing the values. If you have a more modern system that has plenty of RAM then you might want to consider increasing the values, particularly if you have a large library.
  
  

• Out of Memory Error on Database connection
  We have seen cases where an out-of memory exception seems to occur when trying to connect to the Calibre database. This results in a message of the form "DatabaseManager - initConnection: java.sql.SQLException: out of memory" in the calibre2opds log. So far we have only seen this on Windows 64-bit systems but as we do not know why it is occurring then it is possible it can occur in other environments. The fix (if it can be called that) seems to be to give the user running calibre2opds full access permissions on the calibre2opds install folder. Quite why this works is not clear, but the supposition is that in some cases the Java database handling library code must want to create a temporary file there for some reason. The procedure to achieve this is:
  • Use Windows Explorer and find the folder where Calibre2opds was installed (probably something like C:\Program Files\Calibre2opds
  • Right-click on the folder name and select Properties
  • Select the Security tab
  • Click the Edit button
  • Highlight the Users entry to see what permissions are allowed in the table below
  • If it is not already set, then make sure that the Full Control option is selected
  • Hit the OK button
    
    

• Cannot Establish Database Connection
  We have come across this error undrr two different scenarios:
  • The path to your library contains accented (or at least non-ASCII) characters. At the moment ther eis no work-around for this other than to change the location of the library.
  • There is a permissions problem as described in the issue above. The solution is the same as described there
    
    

• Problem with dmg on Mac OSX
  If you get a message when trying to use the downloaded dmg file along the lines of "Maybe there's a problem with this Disk Image. Are you sure you want to open it ? Opening this Disk Image may compromise the security of your computer or cause other problems." then you can safely ignore the error and proceed. Just be certain that you got the dmg file from the official calibre2opds download site as that version is known to be good.This message appears to be an artifact of the fact that the Mac OSX dmg file is created on Windows (using a tool called TransMac) and not under OSX. This is to get around the fact that the calibre2opds developers do not have a Mac and thus cannot produce (or test) the dmg file on a real Mac.
  
  

• The catalog appears to be OK but the covers do not display correctly and/or the download links do not work
  The calibre2opds generated catalog will use the information the Calibre metadata.db file to generate the filenames to be used for covers and downloads.This can cause this type of problem if you use Calibre on Windows to manage your library (Windows ignores the case in filenames) and the web server runs on a system that treats case in filenames as significant. If you changed the case of the author or title in Calibre without actually changing the spelling then you can end up with a case mismatch bewteen the Calibre database and the files on disk.As an example most NAS type boxes that support a web server are running Linux under the covers and Linux treats case as significant.The only solution as this is encountered is to get the names in the Calibre database to exactly match the names on disk including the case of the name. The easiest way to do this is from within Calibre itself using the following procedure:
  • Find the book that is causing a problem. If this is the only book by the given author then the following steps will work. If you have multiple books by the same author and the case mismatch is on the author name rather than the book title look at the TIP below.
  • Select the Edit Metadata option for the book.
  • Edit the author in some way (e.g. add an X on the end) and press OK. This will cause Calibre to rename the book on disk, and after the rename the name will exactly match the entry in the Calibre database.
  • Select the Edit Metadata option again and change the author back to the correct value and press OK. Now with any luck everything will be OK.

TIP: If you have multiple books by the same author exhibiting this problem then an alternative is to find the author in the tag browser and use the option to rename the author there. Follow the same procedure of changing it to a name that does not exist, pressing OK, and then changing it back and pressing OK again.

The disadvantage of this approach is that it can be slightly slower as Calibre might have to do renames on book files that were already correct. However it is the more foolproof method if you are not sure.

• I get a database error trying to generate catalogs.
  This normally means that either you have specified an incorrect path to the Calibre library folder, or that another program has the calibre metadata.db file locked (probably Calibre itself!).
  
  

• In the All Books section of catalogs I find that there are some books listed under Other Books that I expected to be listed under another letter
  Check the Titles in Calibre. It is possible that they start with a leading space.
  
  

• In the Authors section of the catalog I find an author is displayed incorrectly or listed under the wrong letter
  Check the Author Sort field for the books belonging to this author. If one of them is wrong it can cause unexpected results.
  
  

• In the Authors section of the catalog I appear to have the same author listed more than once.
  This can happen if you have two authors with the same value for the 'Author' field in Calibre but have the AuthorSort field is different.
  
  

• When I try to connect to a catalog from Stanza I get a Catalog error message.
  This is generic message that is given if Stanza cannot download a catalog for some reason. Check the URL that you have used and that you do not have a firewall blocking access.A useful way to check things and eliminate Stanza setup as an issue is to use a Browser such as Internet Explorer and give it the URL that you think you should be using. If the URL is correct the browser will display the page from your catalog although it may not work correctly (particularly with Internet Explorer).
  
  

• My ereader shows me the book in the OPDS (xml) catalog but there is no download link for it
  There are two common causes of this issue:
  • You have not got a compatible format of the book in your library.
  • You are using Stanza and you did not select Stanza compatibility mode when you generated your catalog. This is important as Stanza implements an early version of the OPDS standard for catalogs, and does not correctly support the current version of the standard. Setting the Stanza compatibility mode generates a catalog that avoids new features of the standard that Stanza does not handle correctly.
    
    

• I click on a Download link in the HTML catalog and random characters appear.
  Some users find that if they are accessing the HTML version of the catalog via a Web Browser, when they click on the Download link then what appear to be random characters are displayed in the Browser. It has been found that Kindle users trying to download MOBI format books are the most likely to encounter this issue. This occurs because the file type has not been recognized, and the e-book file is being downloaded as if it were a simple text file. Since most e-book formats are stored in a compressed format, this is why you see the random characters. The reason this is happening is that the server is not correctly sending what is known as the mime type to identify the file type that is being downloaded. Calibre2opds does specify this is the download link within the HTML, but many web servers ignore the HTML and require the mime type to be specified in a separate configuration file (the file will be specific to the type of web server). If you encounter this problem then you would need to contact whoever is running the web server to get the mime type correctly specified for the web server in use.
  
  

• The author is not sorted the same way in Calibre2opds catalogs as it is in the Calibre library view
  There are two places in Calibre that the author_sort value can be specified. One is on the author records, and the other is on individual book records. Calibre2opds always uses the author_sort value from the author records.

  Normally the values on the author records and book records will be the same. You can tell if there is a mismatch by going into the Edit Metadata dialog for an individual book. If the value stored for this book is different to that stored for the author then the author_sort field will have a pink background whereas if they agree the background will be green. If you want to make all books for given author use the same author_sort value, then open up the tags pane, find the author in question and right-click to select the Edit author_sort option. When you set the author sort value and press OK then all books by this author will have their individual author_sort fields set to match the setting at the author level.

  If you are using calibre2opds 3.2 or later then there will also be a warning message output to the log indicating that such a mismatch has been spotted and telling you what book(s) is involved.</li>

Gathering Diagnostics

If calibre2opds is not behaving as expected then there is some useful diagnostic information that is available. This information is stored in the calibre2opds configuration folder. The configuration folder is called calibre2opds and will typically be located in the user's home folder although the CALIBRE2OPDS_CONFIG environment variable can specify an alternative location.

To help you with locating this configuration folder, there is an option on the calibre2opds Help menu that can be used to open this folder:

Configuration Settings

The calibre2opds configuration settings are stored in .profile.xml files. These can be loaded and examined using a text editor although the user should never change their contents directly. There will normally be a file called default.profile.xml that is created initially. If you create additional profiles using the Profile menu in calibre2opds then there will be additional .profile.xml files - one per profile.

It can be useful to attach the relevant profile file to any problem you report or bug you raise as it contains the exact settings you had set up in calibre2opds. This can be helpful both in discussing an issue, and also, if necessary, in helping the developers report a problem.

Log Files

Calibre2opds is used on a wide variety of different systems and in vastly different configurations. To help diagnose any problems that might arise in such a system being able to log how the program is running is an important feature. The logging system in calibre2opds is designed in such a way that it can be useful to a wide spectrum of users ranging from the average end-user right through to the developers of the software.

There are some log files that written to the log sub-folder of the calibre2opds configuration (home) folder. They are simple text files and can be examined with any text editor. You can also open the log file for viewing by using the option on the calibre2opds Help menu as shown above.

• calibre2opds.error.log:
  This is intended to catch details of catastrophic errors such as calibre2opds aborting with a Java error. It will normally be absent and/or empty.

• calibre2opds.log: This file contains information about how calibre2opds was running. It can be useful to give a more detailed view about what was happening. 'under-the-covers' while calibre2opds was running.It is useful if this log file is supplied as part of any bug report as well as potentially showing any error messages it also shows what options were selected when a generate run is performed.<p>Calibre2opds using what is known as a 'rolling file' approach to logging where the file calibre2opds.log is the most recent log file, and older ones are identified by adding a number to the name (e.g. calibre2opds-1.log and the larger the number the older the file. To avoid log files accumulating endlessly, there is an absolute limit set in the xmlj2.xml configuration file on how many older log files should be kept (the default is currently set to 10). The 'rollover' is configured by default to happen every time calibre2opds is started, and also if a log file reaches a size of 1MB.

Setting Log Detail

The level of detail that is logged is controlled via a file called log4j2.xml that is stored in the log sub-folder of the calibre2opds home folder. When calibre2opds is started then if this file does not already exist it is created with settings suitable for normal running. This is a standard text file that can be viewed or edited using a text editor program.

Do not be put off by its content - as a user you will not only ever need to make simple changes to this file and do not need to understand how it was initially constructed. Inpractise it is likely that the vast majority of users will never encounter situations where changing the detail level is required, but it is documented here for those rare occasions where it might be requested by the developers.

The logging can occur at various levels. Every message in the log has a field that indicates at what level this particular message is triggered. Each level is significantly more verbose than the previous level. The more verbose levels are also likely to adversely affect performance due to the overheads of generating and writing out the level of information specified.

The levels that are currently available are:

• FATAL
  Hopefully this level will never be seen although it is always active! It indicates that an unrecoverable error has been encountered and Calibre2opds is going to be immediately terminated.

• ERROR
  This level is always active, and is used to indicate errors that Calibre2opds has encountered that do not normally lead to immediate termination of the program.

• INFO
  This is the level at which the default level of logging is run. The output is designed to be meaningful without any special knowledge of the internals of calibre2opds. An average user should find that they could look at the output produced and easily relate it back to their use of the calibre2opds program.

• DEBUG
  This level adds additional information about how calibre2opds is running. It is designed to be meaningful to a user who has some basic technical knowledge of how calibre2opds functions, but is not aware of the detail. This level could be usefully used by end-users who might be trying to look at a problem with using calibre2opds on their system.

• TRACE
  This level is designed to help the programmer sort out the more intractable problems. Much of the output might not be meaningful without reference to the source code of calibre2opds. This level of logging is unlikely to be useful to an end-user so it would normally only be activated if specifically requested by calibre2opds developers or support staff.

Calibre2opds makes use of a publically available Java library known as log4j 2 to handle logging. This logging facility is highly configurable and helps immensely during the development of calibre2opds. The log4j2 facility is also used to provide useful information to the average Calibre2opds user about their own runs. To avoid the need in most cases for the end-user to understand the format of log4j configuration files a number of pre-configured log files are provided with the name indicating the level of detail. These files can be renamed to lo4j.xml to achieve the level of detail indicated. The files that are provided are

• log4j2.xml
  This is the current active configuration file. By default it will be the same as the log4j2.info.xml file.
• log4j2.info.xml
  This is the default settings. It provide information that is likely to be of interest to the average user of Calibre2opds.
• log4j2.debug.xml
  This configuration file will cause more information to be logged that can be useful to debug problems without being so verbose that it is over-whelming.
• log4j2.trace.xml
  This is a configuration file that logs a lot of information about the internal workings of Calibre2opds. The logged information would normally only be of use to a developer who is familiar with the internal workings of Calibre2opds.
• log4j..trace.noCachedFileTracing.xml
  This is even more verbose than the default trace level above as it logs information about every file access. Calibre2opds has an internal file cache system that is used to increase performance. This level of detail would normally be too verbose to be useful but might be used if it was thought there might be an error in the cache handling within calibre2opds

Reporting Problems

If you encounter an issue there are a number of places you can go to get assistance.

Forum

The first port of call will normally be the dedicated calibre2opds forum at GetSatisfaction. this is where you can make contact with other calibre2opds users and also the calibre2opds development team.

Here you can look through the issue that previous users have raised to see if your problem had been raised and answered previously. If not then start a new topic giving as much information about your issue as you can. Do not be impatient and expect an immediate reply as users are scattered around the world in a wide variety of time zones.

Although it is no longer the prefered venue for calibre2opds discussions, any calibre2opds issues raised in the Calibre forum at MobileRead will normally also be seen and get a response.

GitHub

The development side of calibre2opds is run as a project on the GitHub site. This is where the source repository for calibre2opds is currently held and interested users are able to view or download the source from there. Those who fancy themselves as programmers can also submit patches to be reviewed and (if accepted) to be incorporated into future releases of calibre2opds.

YouTrack tracker

This is the place to report bugs in the calibre2opds software. The site includes a bug tracking system so that once a bug has been entered its status can be viewed and tracked. Using the bug tracking system makes sure that it is not forgotten about and makes sure that the developers are aware of it.

Creating a Problem Report

In simple terms whenever you want to report a problem, the key information is to provide as much information as possible so that if the solution is not immediately obvious, steps can be taken to try and reproduce your problem in a controlled environment.

Examples of the sort of information tha it can be useful to provide are:

• The exact version of calibre2opds that you are using. Both the version and rev level are useful - these are displayed on the main screen

• The operating system (and if relevance the version number of the OS) that you are using

• The steps necessary to reproduce the problem.

• The calibre2opds log files mentioned earlier. These provide information both about the generation time settings that were in use, and how far a generate process managed to run. Note however that these log files can be over-written if you run calibre2opds again, so make sure you take copies of the one illustrating the problem before running calibre2opds again.

• Screen shots can help illustrate many problems. The method of obtaining these is Operating System dependant so you need to look in your Operating System documentation to produce a screen shot.

• In extreme cases you might be asked to provide a copy of your Calibre metadata.db file if it is suspected that your problem is:data dependant. You need to consider whether providing this would cause yoU any privacy concerns. Hopefully it would not as for most people the information in the Calibre database is not personal. You can also mark such a file when attached to a Problem report as only being visible to Calibre2opds developers which stops it being publically visible.

Mime Types

The Mime type is used during file download to identify the type of file that is being sent. For reference purposes, the following table shows examples of the mime types that Calibre2opds specifies for different common file types:

File Type	Mime type
AZW	application/octet-stream
EPUB	application/epub+zip
LIT	application/x-ms-reader
LRF	application/x-sony-bbeb
LRX	application/x-bbeb-book
MOBI	application/x-mobipocket-ebook

The list above is by no means the complete list of mime types recognized by Calibre2opds, and new ones are periodically added. When you generate a Calibre2opds catalog then a text file called mimetypes.txt is output to the root of the catalog that contains the full list of mime types that are currently understood by Calibre2opds.

Calibre2opds provides a capability for the user to extend or change the list of mime types themselves if they have a file type in their Calibre library for which Calibre2opds does not currently know the relevant mime type.