magic with TOC: edit or create a TOC for your documents or ebooks

Hope you enjoyed polishing and shining your e-books. Now let's look at more heavy duty stuff. The last post already detailed how you can get the "Edit TOC" button on the main toolbar. It is the rightmost button on the main toolbar in the figure below.

Create TOC:

If you have an e-book without a  Table of Contents (TOC),  but it is well structured with headings calibre allows you to create a TOC in a few simple steps.Consider the following "Example Ebook":

As you can see the e-book has no TOC, although it has headings and sub headings. Select the e-book in the main calibre window and click the "Edit TOC" button. A new window opens with the following options in the right side menu.

Click on "Generate TOC from all headings. The following TOC shows up in the editor window:

It has a well constructed TOC. Sometimes the headings in the ebook file are not well marked and some may be missing. In that case you can edit the generated TOC as explained in the next section.

Here is what the above generated TOC looks like in the calibre viewer:

Edit TOC:
So here I will start with a book that has an unsatisfactory TOC and play around with some of the available tools to make it better. This is just an example to show you some of the things you can do.

The above figure is a snapshot of the calibre viewer with the TOC of the e-book displayed. This TOC is messy as some chapters are subsets of others and although the chapters have names in the book, the TOC only lists chapter numbers. So lets see what we can do to spruce it up.

Select the e-book in the main calibre window and click the "Edit TOC" button. Then the above window opens up. Now as you can see Chapter II to Chapter XI are a part of Chapter I. Now say you want to make them independent chapters not belonging to Chapter I then select Chapter I and click the "Flatten this entry" button on the right hand side menu. Once you do this you will see:

If you now click "OK" and view the ebook the TOC looks like

As you can see in the two above figures Chapter II to Chapter XI are in the same level as Chapter I and not a part of it. But now say you want to make Chapter I to Chapter XI all members of Part I of the book. That too can be done.

In the "Edit TOC" window shown two figures above select "Preface". Then on the menu on the right, click "New entry below this entry". This takes you to the following window:

Move the mouse and position it just above Chapter one and left click. On the right hand side replace "(Untitled)" with "Part I" and click "OK". Now this is what you see.

To make Chapter I to Chapter XI  members of Part one just drag and drop them in to part one in the correct order. Then this is what you see.

Now if you click "OK" and open the book in the viewer you will see:

You can also edit the text for any of the entries in the TOC by double clicking the entry in the TOC editor window. As you can see below I have edited "Chapter I" to the name of the chapter in the book.

The following figure shows the TOC in the viewer with all the chapters in Part I with their proper names.

This articles outlines some of the things you can do with "Edit TOC". There are a few other options too. This should help you get started. Then enjoy tweaking and playing around with all the options. Feel free to leave comments about other interesting things you tried.

this is the way to polish your e-books

Hi
It has been a while but again calibre has some great new features. So let us take a look at them. Here I will discuss "Polish Ebooks" introduced in 0.9.23 and next time "Edit TOC" introduced in 0.9.25. Note that polishing is only supported for books in AZW3 and EPUB formats. Hopefully any bugs that were present to start with have been ironed out. Your feedback is always welcome and since this is a fairly new feature, click here to report any bugs.

Adding the buttons:
To start with lets get the "Polish" and "Edit TOC" buttons, so we can click on them.. Go to "Preferences", click on "Toolbar", click on the button that says "Click to choose toolbar or menu to customize". In this case I have chosen "Main Toolbar" because I want the icons for "Polish" and "Edit TOC" to appear in the main toolbar so I can access them easily. If you think you wont use them too often you can have them on a secondary optional toolbar or wherever is convenient for you.
So after I click main toolbar the following window opens up:

On the left menu of available actions you will see the "Polish" and "Edit TOC" buttons highlighted by red rectangles in the above figure. They are towards the bottom of the menu so you may have to scroll down. Select them both. then click the right pointing blue arrow (circled in red in the above figure). This will move these icons from the available actions list to the current actions list. Click "Apply" on the top left corner and close your Preferences window. Now the added buttons will be included in the toolbar you chose. In this case in the main toolbar as shown below:

Polishing Options:

Select all the e-books to be polished in the same way and then click the polish icon in the main toolbar. The following window opens up:

The available options are outlined here.
You can:

• Subset all embedded fonts
• Smarten punctuation
• Update metadata
• Update cover
• Add a metadata page at the start of the book
• Remove a previously inserted metadata page

Many of these options were available earlier too, but to implement them you would have to convert the book again which for some e-books can be slow and CPU intensive. Now you can just spruce them up quickly because  polishing, unlike conversion, does not change the internal structure/markup of your book, it performs only the minimal set of actions needed to achieve its goals.
The "About" button next to each action gives details about exactly what the particular polishing tool achieves.

Subset all embedded fonts and Smarten punctuation:

This is just copied from the about section of "Subset all embedded fonts". The information is quite clear and I had nothing to add to it:

Similarly smarten punctuation simply converts plain text dashes, ellipsis, quotes, multiple hyphens, etc. into their typographically correct equivalents.
Caveat: The algorithm for "Smarten punctuation" can occasionally generate incorrect output.

Update metadata and Update cover in book files:

When you click "Edit Metadata" in calibre and change the metadata or the cover of a book the information is saved in a file called metadata.db in your calibre library. This information is only transferred to the particular book file when you convert the book or export it. However now this information can be updated in the book files by simply choosing the appropriate options in "Polish". Note there are separate buttons for updating the cover and for updating the rest of the metadata. The metadata that is updated will depend on what metadata the e-book format supports. Unfortunately many e-book formats do not support all the metadata that calibre allows you to store.

The following is a screen shot of calibre before "Update cover in book files" was done:

The e-book had been downloaded from Project Gutenberg.  then the metadata was edited to add the cover you can see on the right information panel of the main calibre window. But the polishing was not done so the the cover displayed when viewed was the generic cover page Project Gutenberg uses for all books.

Next we ran the "Update cover in book files" under "Polish" and voila,

the chosen cover is now the first page in the viewer.

Add metadata as book jacket page:

This allows you to add a book jacket containing all the metadata you added under "Edit Metadata" in calibre, including title, author, publisher, tags, comments, rating, series information etc. The book jacket page is always the very first page and precedes the cover page if there is one.

The following shows the book jacket for the book we have been using as an example so far:

The last option in "polish" is to remove the previously added book jacket in case you did not like it.

Coming soon:
In a week or two I will discuss the "Edit TOC" feature added in 0.9.25. It may still have a few bugs so let's give it some time to be straightened out. Also I have to go learn about it.

Hope you enjoyed this post. Have fun polishing away.

calibre 0.9.0 is here!

The long awaited calibre 0.9.0 is here! This is the link to the page documenting all the improvements since 0.8.0.




Some of the important improvements I will discuss here at length are:

1. Improved support for android phones and tablets 
• MTP support
• Calibre Companion
2. Viewer
• Display of mathematics in e-books
• Page mode
• Table of contents panel now tracks the current position in the books
3. Searching for accented characters
4. Wiki Reader plugin
5. Some other popular features introduced after 0.8.0

Improved Support for Android Phones and Tablets

Over the last year we have seen many new android phones and tablets flood the market. These devices double as readers and are used by many to read news and ebooks. Calibre has been adding support for these devices as and when they have appeared. While most earlier android phones and tablets presented a drive letter when connected via USB to the computer, many of the newer ones do not including the popular new Nexus 7. For those android devices that do present this drive letter, calibre  has been able to detect the device when it is connected to the computer and transfer e-books to and from it. It can also connect to it wirelessly using the over the air method explained  here. This wireless method requires setting up the calibre content server.

MTP Support

However many new android devices do not present a drive letter when connected by USB to the computer. These devices use something called Media Transfer Protocol (MTP) for data transfer. Prior to calibre 0.8.53, calibre could not detect these devices via the wired USB connection. Also some people had trouble setting up the calibre content server to connect to them wirelessly. We realized it was important to improve the support for these devices.

Now calibre can connect via USB to even those devices that do not present a drive letter. A lot of you were waiting for this feature and now it is here. You can use calibre to manage your ebooks on your android phones or tablets just as easily as you did on your sony readers, kindles, nooks etc.Unfortunately we have not been able to get this to work with OS X but it does work with Windows and Linux.

When you connect an MTP device like the Nexus 7 to calibre, a device icon like the one in the above picture shows up in the tool bar. You can see what books are on the device by clicking "Show books in main memory of this device". All books in your library which are also on the device will be marked with a green check mark when the device is connected by USB to your computer. You can right click on any book in your library and it will give you the option to send to device.

Calibre Companion

Many of you have also been asking for a calibre app for wirelessly connected android devices. For this we recommend a new app called  Calibre Companion. Charles Haley, who has made significant contributions to calibre's code, is one of the developers of Calibre Companion. 

According to the Calibre Companion page on Google Play:

Calibre Companion is two applications in one.

• The first: Calibre Companion supports connecting to calibre over WiFi and be detected by calibre as a device.
• All calibre device operations are supported, including sending books and metadata, cover thumbnails, deleting books, sending news, and plugins like Reading List.
• You can specify where different book formats are to go on your device, e.g., epubs in one folder, pdfs in another, and mobis in a third.
• Calibre Companion can update the metadata for the books on every connect, ensuring the information is up to date
• Most setup is on the device, making it easy for different users in the same household to have different device configurations, or to use more than one computer running calibre.
• The second app: Calibre Companion is a library organizer for your device. You can:
• Sort books by title, author, series, date changed or read, with both ascending and descending sorts.
• Group books by title, author, tags, series, publisher, and most calibre custom column types; including optional navigation by first letter.
• View a "book jacket" showing all the details of a book, including calibre's custom columns.
• Launch your book reader application to read a book (works with most reader applications).
• Search for books using calibre-like search syntax.
• Scan your library folders for books not in Calibre Companion's database and adding them. This process uses calibre to help get the highest quality metadata available.

This app is supported on the calibre devices forum at MobileRead.

Calibre Companion is sold for $2.99 all the proceeds of the sale go to  a company called MultiPie Ltd. Calibre Companion is not made by calibre's primary developer Kovid Goyal and he does not receive any of the income generated from it's sale.

Improvements to the calibre viewer

Calibre is primarily an ebook format conversion and library management program. But it has an inbuilt viewer. The viewer allows you to read your ebooks even if you do not own a dedicated reading device or if you just want to read on your computer. The calibre viewer supports a myriad of formats and we have already done a detailed blog post on its various features. The last year has seen a number of significant and popular improvements to the calibre viewer.

Display of mathematics in ebooks

Although ebooks have been excellent for regular text there has been some problems with including math in ebooks. The calibre viewer can now display math embedded in ebooks (ePub and HTML files) extending it's functionality to reading scientific and technical content. To create an ebook with math you can typeset the math directly with TeX or MathML or AsciiMath. The calibre viewer uses the excellent MathJax library to implement this feature. For more information on this feature click here. 

Caveat: The calibre viewer supports displaying math, but this does not mean that your ebook reader can display the math. 


Here is an example. This HTML code has been converted into epub by calibre. The screen shot of the epub file on the calibre viewer follows.

Page mode

The calibre viewer now has a page mode. In this mode the scrolling is disabled and the content is broken up into discrete pages. The bottom or top line is no longer cut off. The white page turning arrows inside the purple circles on the left toolbar have to be used to turn pages. 

To activate the page mode you have to click on the following icon on the top right corner of the viewer:

To disable page mode click on the same icon again. The following picture shows a comparison of the viewer display in scroll mode (on the left) v/s page mode (on the right).

As you can see, in scroll mode the top line is cut off while in page mode it is not.

Table of contents tracks the current position

The table of contents on the left side of the calibre viewer tracks the current position in the book. When the viewer displays a particular page in the book, the section to which that page belongs is highlighted in bold in the table of contents panel. For example see figure below:

The content displayed in the viewer is from the chapter "JEEVES AND THE UNBIDDEN GUEST" which is in bold in the table of contents panel.

Searching for accented characters

In the main calibre window you can search for books by entering into the search bar any word included in the book's metadata. If any word in the metadata has an accented character like "ë" for example you can just enter "e" in the search bar. This is not just true for author names but for all entries in the metadata. The following picture shows how you can search for an ebook titled "EmilyBrontë " by just entering "bronte" in the search bar, i.e simply "e" instead of "ë".

Wiki Reader Plugin

The Wiki Reader plugin allows you to create an ebook of pages from Wikipedia by entering the URLs of interest. You can give the ebook a title of your choice. If you do not enter a title it will just use the title of the article corresponding to the first URL. The ebook will have the same table of contents as on the wiki page. The author is set to Wikipedia and the cover page has the wikipedia logo. For details on usage click here. 

Here is an example of an ebook created with the wikipedia pages for e-book, e-book reader, DRM and calibre where the viewer is displaying the first page of the calibre wikipedia article.

As you can see the highest level of the table of contents (TOC) just has the titles of the four articles. The sub-levels are generated from the TOC of the particular wiki page. Below you can see the TOC on the calibre software wiki page and compare it with the sections in the TOC under calibre (software) in the above picture.

Other similar plugins include Recipe Reader, Medical Reader and Hymn Reader. The Recipe Reader allows you to make an ebook of recipes from cooks.com by entering the URLs of interest from the cooks.com website. The Hymn Reader allows you to make an ebook of hymns from Hymnwiki by entering the URLs of interest from the Hymnwiki website.

Other popular features introduced after 0.8.0

A number of popular and useful features introduced in to calibre after the release of 0.8.0 have already been discussed in earlier blog posts. These include:

• Column coloring: Column coloring allows you to set up conditions to have certain entries in certain columns appear a certain color.
• Calibre portable:This version of calibre can be put on to a USB stick and carried around with you. The USB stick would contain both the calibre installation as well as the calibre library. It can be then plugged into any windows computer and used to access the library on the stick with calibre.
• Auto clean up of news downloads: This is a powerful feature that enables lay users to make custom recipes. You may be interested in making a single news recipe that has RSS feeds from different blogs and news sources that you visit. This can be done quite easily with "Auto clean up".

Some other popular features introduced post 0.8.0 include:

• Support for KF8 the new kindle format
• An improved full screen mode of the calibre viewer
• A languages entry in metadata

To summarize 0.9.0 has lots to explore and enjoy. 

I have not written a blog post in a while. This is mainly because most of the important topics have already been covered. But as and when something interesting comes up I will try to cover it.

calibre portable: A great holiday gift

For a very long time calibre was tied up to the computer on which it was installed. Although the calibre content server allowed for access to the calibre library through a browser, it still required the computer on which calibre was installed to be on and accessible through the internet. But since version 8.5 a new portable calibre build was made available by Kovid for your convenience. This version of calibre can be put on to a USB stick and carried around with you. The USB stick would contain both the calibre installation as well as the calibre library. It can be then plugged into any windows computer and used to access the library on the stick with calibre.

To download calibre portable go to this page (You can also get here by going to the usual calibre download page here and then clicking on the usb stick icon shown in the figure below), where there is a link to download calibre portable. When you click on the link you will be prompted to save a zipped file. Once you have downloaded it and unzipped it transfer the contents to a USB stick. After you have done this the USB stick should contain a folder named "Calibre Portable". That folder should contain three sub-folders named "Calibre", "Calibre Library" and "Calibre Settings" and an executable file called "calibre-portable.exe".

To include your current calibre library in calibre portable just copy the entire contents of your current calibre library folder on your computer to the "Calibre Library" folder in the "Calibre Portable" folder on the USB stick. Now you are all set up, so safely remove the USB stick. To run calibre portable on another computer; this part only works on windows machines; plug the USB stick in to the windows computer and double-click on the calibre-portable.exe file mentioned above. calibre portable will open up and its ready for use. It has all the features any other calibre installation has. The only difference is it will always start up with the library called "Calibre Library" in the "Calibre Portable" folder. You can of course switch the library while using it but the next time you start calibre portable it wont remember the library switch and will again start up with the library called "Calibre Library" in the "Calibre Portable" folder. But you can manually switch the library each time.



calibre portable as a holiday gift:
calibre portable makes for a great personal and affordable christmas gift. Just go buy a USB stick, 1GB is quite enough for the purpose. Follow the steps outlined above to get calibre portable on to it. calibre portable is absolutely free. Then taking into consideration the likes and dislikes of the recipient there are many excellent places to get books to populate the library. For the classics there is Project Gutenberg, where a huge number of public domain ebooks are available as well formatted ebooks absolutely free. Then there are a number of sites that sell their ebooks DRM-free so your recipient can make the most of this gift. These sites include Baen for speculative fiction, Carina Press for romance, Smashwords, BeWrite Books,  XinXii and many more for a variety of genre.

Following image is courtesy  this site.

calibre maintains its own catalog of DRM-free ebooks from various sources and it is made available through its website Open Books. Open books now has a collection of over 2825 ebooks to choose from and features content from over 1095 authors and from 35 stores. Open books includes over 150 free ebooks and over a 1000 ebooks priced less than $3.00. Note that open books only catalogs DRM-free ebooks from various sites and provides links to them. Open Books was set up to promote authors of DRM free ebooks but Open Books does not sell any ebooks.

So get together some ebooks that your recipient would like and put it into the "Calibre Library" folder in the "Calibre Portable" folder. You can even use calibre to spruce up the metadata and add covers to make all the books look good. You can also add a couple of news magazines your recpient would be interested in using the calibre "Fetch News" feature for free. This makes a great gift, very personal, useful and can be treasured for a long time to come.

Use calibre portable to be a thoughtful Santa and Happy Holidays to all of you. The next calibre blog post will probably be early in the new year. I will be decreasing the frequency of the blog posts since a lot of the major topics have been covered. Please use the comments section to make requests for topics you would like covered.

calibre look and feel

calibre allows you to customize its Graphical User Interface (GUI) in many different ways so that you can have a GUI that suits your particular needs and tastes. In this blog post I will point out some useful tweaks that can be easily implemented and focus mainly on column coloring which is a useful way to identify certain books in your collection.

To adjust the look and feel of the calibre GUI click on Preferences in the main tool bar. In the new window that opens click on the Look and Feel icon in the first row under Interface. The following window appears.

Main Interface: The first tab in the above window is Main Interface. Here you can choose the user interface layout. By default this is set to Wide, in which case book details along with the cover appears in the rightmost column and the cover browser is above the list of books in the main calibre window. However this can be set to Narrow as shown in the figure above and then the book details with cover appear at the bottom and the cover browser on the right as shown in the figure below. You can click the little brown book icon at the bottom right corner of the screen to make the book details disappear altogther and then click the brown book icon again to make them reappear.

You can click the little curved arrow icon at the bottom right corner to mahe the cover browser disappear or reappear. The main interface tab also allows you to choose the icon sizes; small, medium or large as well as off (no icons, only the text under the icons is visible and can be clicked on).

Book Details: The next tab is book details. As shown in the figure below here you can choose what details are to be displayed in the book details section of the main window.

calibre's tag browser has been discussed in some detail in an earlier blog post.

Cover Browser: The fourth tab is the cover browser. This allows you to choose if you would like to have the cover browser displayed within the main calibre interface or in a separate window as shown in the figure below. You can also choose to have this window appear in full screen mode and toggle the full screen mode using the F11 key.

Column Coloring: As shown in the figure below the last tab under Look and Feel is column coloring. Here you can set up conditions to have certain entries in certain columns appear a certain color.

When you click on the Add Rule button the following window appears which allows you to set the entries in a column to be a particular color provided the entries of some other column satisfy some condition by taking certain values.

For example, when we add the following rule (make title blue when genre is news)

we see

but even if we hide the genre column we can still tell which titles belong to the news genre because they are in blue as shown below.

Consider another example where all the titles are made maroon when the genre is not specified as shown in the figure below. This kind of rule would be useful for you to identify books where metadata is incomplete when you are managing your library.

Hope you found these tips useful. See you again in about a week.

calibre custom columns

The main calibre window has a default set of columns that most people find useful in organising their ebook collection. These include Title, Author, Date, Size (of file in MB), Ratings, Tags , Series, Publisher and Published. Although these columns are sufficient for many users, most calibre features are made flexible to accommodate unusual needs and whims. The set of columns in the main window can be easily customized to include those columns that you find most useful. This demo video (the part between approximately 2 and 4 minutes of the 14 minute long video) illustrates how to manipulate the contents of columns as well as how to delete unwanted columns and add custom columns.


Manipulating the columns: Depending on your screen size only a few columns can be made visible so you will only want to include columns that are useful to you. If you need more columns than fit in your screen, you can move the most useful ones to the left so they are always visible. The order of the columns can be changed by left clicking on the column title and dragging it to the left or right of other column titles. As shown in the figure below you can right click on the column title to see ways in which you can manipulate it.

You can hide the column if it is not useful to you. You can also sort the entries in your library in ascending or descending order of the entries in a particular column like the rating; having the highest rated bpooks on top and the lowest rated at the bottom of your list. This can also be used to arrange books in alphabetical (or reverse alphabetical) order of author or title. You can also implement this by simply left clicking on the title of the column you want to sort by. If you want the order reversed just click on the column title again. If the entries are arranged in descending (ascending) order of the entries in a particular column a little grey downward (upward) pointing arrow appears next to the column title; like for the Rating column in the figure above.
For a text type column like author or title you can choose the the text to be left, right, or center aligned. You can manipulate the width of the columns by hovering the mouse between two column titles, once the left-right double sided arrow icon shows up you can drag it left or right to adjust the width.

 

Adding custom columns: To add a custom column click on Preferences in the main calibre tool bar and in the new window that opens click "Add your own columns" in the first row of icons under Interface. The following window appears.

The columns marked with an "x" are the visible columns in your main calibre window. You can select or unselect columns here to choose which ones should be visible. To add a column not on the list click the green "+" sign button on the right and the following window appears.

First you must choose the nature of entries in your custom column. The figure above shows the available data types including integer numbers, text, binary yes and no entries, dates, floating point numbers, etc. This blog post will illustrate examples of two different data type columns.

Example Binary Yes/No type column: Here we are going to create a binary Yes/No type column with column heading "Read", look up name "read" and column type Yes/No. The entry Yes in this column indicates the book is read and No indicates the book is not yet read. After filling the entries and clicking OK in the window in the above figure a new column appears with title "Read" in the main calibre window. By double clicking in the position of the entries of this column a drop down box opens that allows you to select "Yes" or "No" or leave it blank as shown in the figure below. So you can now mark the read status of the ebooks in you collection as done for a few in the figure below. A green tick indicates Yes or read and a white cross in a red circle indicates No or not read. You can also leave the read status blank.

Example Text with fixed set of permitted values type column: We will use this type of column to indicate the genre of the book. The figure below shows how we have chosen the values and corresponding colors for the entries in this column.

Setting the column up as shown above will create a column titled Genre with allowed entries Adventure, Mystery, Classic, Humour, Play and News as shown below. The adventure entry will always be in black the Mystery entry in blue etc.

 To fill the entries of this column as before double click on the column entry and a drop down box will show up with the allowed entries as shown in the figure above.  The figure above also shows a few books with this entry filled in and the News entry is in red while the Humour entry is in green. The colors will help quickly distinguish the different genre at a glance.

Hope you have fun playing around with various types of custom columns this week and find it useful. See you again in about a week.