Thursday, December 15, 2011

calibre portable: A great holiday gift

For a very long time calibre was tied up to the computer on which it was installed. Although the calibre content server allowed for access to the calibre library through a browser, it still required the computer on which calibre was installed to be on and accessible through the internet. But since version 8.5 a new portable calibre build was made available by Kovid for your convenience. This version of calibre can be put on to a USB stick and carried around with you. The USB stick would contain both the calibre installation as well as the calibre library. It can be then plugged into any windows computer and used to access the library on the stick with calibre.

To download calibre portable go to this page (You can also get here by going to the usual calibre download page here and then clicking on the usb stick icon shown in the figure below), where there is a link to download calibre portable. When you click on the link you will be prompted to save a zipped file. Once you have downloaded it and unzipped it transfer the contents to a USB stick. After you have done this the USB stick should contain a folder named "Calibre Portable". That folder should contain three sub-folders named "Calibre", "Calibre Library" and "Calibre Settings" and an executable file called "calibre-portable.exe".



To include your current calibre library in calibre portable just copy the entire contents of your current calibre library folder on your computer to the "Calibre Library" folder in the "Calibre Portable" folder on the USB stick. Now you are all set up, so safely remove the USB stick. To run calibre portable on another computer; this part only works on windows machines; plug the USB stick in to the windows computer and double-click on the calibre-portable.exe file mentioned above. calibre portable will open up and its ready for use. It has all the features any other calibre installation has. The only difference is it will always start up with the library called "Calibre Library" in the "Calibre Portable" folder. You can of course switch the library while using it but the next time you start calibre portable it wont remember the library switch and will again start up with the library called "Calibre Library" in the "Calibre Portable" folder. But you can manually switch the library each time.



calibre portable as a holiday gift:
calibre portable makes for a great personal and affordable christmas gift. Just go buy a USB stick, 1GB is quite enough for the purpose. Follow the steps outlined above to get calibre portable on to it. calibre portable is absolutely free. Then taking into consideration the likes and dislikes of the recipient there are many excellent places to get books to populate the library. For the classics there is Project Gutenberg, where a huge number of public domain ebooks are available as well formatted ebooks absolutely free. Then there are a number of sites that sell their ebooks DRM-free so your recipient can make the most of this gift. These sites include Baen for speculative fiction, Carina Press for romance, Smashwords, BeWrite BooksXinXii and many more for a variety of genre.

Following image is courtesy  this site.



calibre maintains its own catalog of DRM-free ebooks from various sources and it is made available through its website Open Books. Open books now has a collection of over 2825 ebooks to choose from and features content from over 1095 authors and from 35 stores. Open books includes over 150 free ebooks and over a 1000 ebooks priced less than $3.00. Note that open books only catalogs DRM-free ebooks from various sites and provides links to them. Open Books was set up to promote authors of DRM free ebooks but Open Books does not sell any ebooks.

So get together some ebooks that your recipient would like and put it into the "Calibre Library" folder in the "Calibre Portable" folder. You can even use calibre to spruce up the metadata and add covers to make all the books look good. You can also add a couple of news magazines your recpient would be interested in using the calibre "Fetch News" feature for free. This makes a great gift, very personal, useful and can be treasured for a long time to come.

Use calibre portable to be a thoughtful Santa and Happy Holidays to all of you. The next calibre blog post will probably be early in the new year. I will be decreasing the frequency of the blog posts since a lot of the major topics have been covered. Please use the comments section to make requests for topics you would like covered.

Saturday, December 3, 2011

calibre look and feel

calibre allows you to customize its Graphical User Interface (GUI) in many different ways so that you can have a GUI that suits your particular needs and tastes. In this blog post I will point out some useful tweaks that can be easily implemented and focus mainly on column coloring which is a useful way to identify certain books in your collection.

To adjust the look and feel of the calibre GUI click on Preferences in the main tool bar. In the new window that opens click on the Look and Feel icon in the first row under Interface. The following window appears.




Main Interface: The first tab in the above window is Main Interface. Here you can choose the user interface layout. By default this is set to Wide, in which case book details along with the cover appears in the rightmost column and the cover browser is above the list of books in the main calibre window. However this can be set to Narrow as shown in the figure above and then the book details with cover appear at the bottom and the cover browser on the right as shown in the figure below. You can click the little brown book icon at the bottom right corner of the screen to make the book details disappear altogther and then click the brown book icon again to make them reappear.



You can click the little curved arrow icon at the bottom right corner to mahe the cover browser disappear or reappear. The main interface tab also allows you to choose the icon sizes; small, medium or large as well as off (no icons, only the text under the icons is visible and can be clicked on).

Book Details:
The next tab is book details. As shown in the figure below here you can choose what details are to be displayed in the book details section of the main window.



calibre's tag browser has been discussed in some detail in an earlier blog post.

Cover Browser: The fourth tab is the cover browser. This allows you to choose if you would like to have the cover browser displayed within the main calibre interface or in a separate window as shown in the figure below. You can also choose to have this window appear in full screen mode and toggle the full screen mode using the F11 key.

Column Coloring:
As shown in the figure below the last tab under Look and Feel is column coloring. Here you can set up conditions to have certain entries in certain columns appear a certain color.




When you click on the Add Rule button the following window appears which allows you to set the entries in a column to be a particular color provided the entries of some other column satisfy some condition by taking certain values.



For example, when we add the following rule (make title blue when genre is news)



we see



but even if we hide the genre column we can still tell which titles belong to the news genre because they are in blue as shown below.



Consider another example where all the titles are made maroon when the genre is not specified as shown in the figure below. This kind of rule would be useful for you to identify books where metadata is incomplete when you are managing your library.



Hope you found these tips useful. See you again in about a week.

Friday, November 25, 2011

calibre custom columns

The main calibre window has a default set of columns that most people find useful in organising their ebook collection. These include Title, Author, Date, Size (of file in MB), Ratings, Tags , Series, Publisher and Published. Although these columns are sufficient for many users, most calibre features are made flexible to accommodate unusual needs and whims. The set of columns in the main window can be easily customized to include those columns that you find most useful. This demo video (the part between approximately 2 and 4 minutes of the 14 minute long video) illustrates how to manipulate the contents of columns as well as how to delete unwanted columns and add custom columns.


Manipulating the columns: Depending on your screen size only a few columns can be made visible so you will only want to include columns that are useful to you. If you need more columns than fit in your screen, you can move the most useful ones to the left so they are always visible. The order of the columns can be changed by left clicking on the column title and dragging it to the left or right of other column titles. As shown in the figure below you can right click on the column title to see ways in which you can manipulate it.




You can hide the column if it is not useful to you. You can also sort the entries in your library in ascending or descending order of the entries in a particular column like the rating; having the highest rated bpooks on top and the lowest rated at the bottom of your list. This can also be used to arrange books in alphabetical (or reverse alphabetical) order of author or title. You can also implement this by simply left clicking on the title of the column you want to sort by. If you want the order reversed just click on the column title again. If the entries are arranged in descending (ascending) order of the entries in a particular column a little grey downward (upward) pointing arrow appears next to the column title; like for the Rating column in the figure above.
For a text type column like author or title you can choose the the text to be left, right, or center aligned. You can manipulate the width of the columns by hovering the mouse between two column titles, once the left-right double sided arrow icon shows up you can drag it left or right to adjust the width.

 

Adding custom columns: To add a custom column click on Preferences in the main calibre tool bar and in the new window that opens click "Add your own columns" in the first row of icons under Interface. The following window appears.



The columns marked with an "x" are the visible columns in your main calibre window. You can select or unselect columns here to choose which ones should be visible. To add a column not on the list click the green "+" sign button on the right and the following window appears.





First you must choose the nature of entries in your custom column. The figure above shows the available data types including integer numbers, text, binary yes and no entries, dates, floating point numbers, etc. This blog post will illustrate examples of two different data type columns.

Example Binary Yes/No type column: Here we are going to create a binary Yes/No type column with column heading "Read", look up name "read" and column type Yes/No. The entry Yes in this column indicates the book is read and No indicates the book is not yet read. After filling the entries and clicking OK in the window in the above figure a new column appears with title "Read" in the main calibre window. By double clicking in the position of the entries of this column a drop down box opens that allows you to select "Yes" or "No" or leave it blank as shown in the figure below. So you can now mark the read status of the ebooks in you collection as done for a few in the figure below. A green tick indicates Yes or read and a white cross in a red circle indicates No or not read. You can also leave the read status blank.








Example Text with fixed set of permitted values type column: We will use this type of column to indicate the genre of the book. The figure below shows how we have chosen the values and corresponding colors for the entries in this column.





Setting the column up as shown above will create a column titled Genre with allowed entries Adventure, Mystery, Classic, Humour, Play and News as shown below. The adventure entry will always be in black the Mystery entry in blue etc.


 To fill the entries of this column as before double click on the column entry and a drop down box will show up with the allowed entries as shown in the figure above.  The figure above also shows a few books with this entry filled in and the News entry is in red while the Humour entry is in green. The colors will help quickly distinguish the different genre at a glance.

Hope you have fun playing around with various types of custom columns this week and find it useful. See you again in about a week.

Friday, November 18, 2011

plugins continued ...

The last blog post talked about plugins in general and illustrated the workings of the duplicate finder plugin in detail. Today I will point out a few other useful plugins. To obtain these plugins follow the same procedure as in the section "Getting the Find Duplicates plugin" in the previous blog post, but with the name of the plugin replaced with the one you are looking for. You may encounter a window in the process that asks you where you would like to install the plugin icon. In the following I have assumed that once you install the plugin the icon is available in the main calibre tool bar. If your screen size is small or if you have too many icons in the main tool bar not all of them will be visible. To see them all click on the little ">>" sign at the extreme right corner of the main tool bar.

Generate Cover Plugin: This plugin was developed by Grant Drake. Click here to go to the mobileread forum on this plugin. The plugin helps you generate custom covers for those books that you cant find a cover for via download metadata or if you simply don't like the available covers. It can also be used to make covers for personal documents like collections of letters or magazine articles or news downloads. The plugin uses available metadata to create the cover. The following figure shows the icon for the plugin you will see in your toolbar.





Select the ebook, you want to generate a cover for, in the main calibre window. Then click on the above icon. The following window appears.



The right side of the window shows the default cover the plugin generates. But this can be customized to your taste. The first tab in this window is "Settings". Below it you will see "Saved Settings" where there will be the "Default" option that will have all the settings of fonts, dimensions, images and contents of the cover shown on the right. You can either add a new setting or just change the default settings. To create a new setting click on the green "+" sign on the right hand top corner of "Saved Settings". A window will open that will allow you to choose a name for the setting. In this example I have chosen "antique" as you can see in the figure below. After "Saved Settings" is "Select Image". Here you can click on the green "+" sign to choose an image stored somewhere on your computer. The next tab "Fonts" allows you to choose the font sizes for the title, the author, the series and any other custom text you want to display on the cover. Under this tab you can also choose the background color, the border color, the text color, if the text should have a border and if it does what color the text border should be. The next tab is "Dimensions", under which you can choose the cover height and width. You can also choose margin sizes and border thickness of both image and cover. Finally the last tab is "Contents". Here you can choose what metadata to display in the "Field Order" section. In the following figure title, author and image are included. You can also choose the order in which these are displayed. For example in the figure below author is moved below image, while it is above image in the figure above. To do this I just selected "Author" under the "Field Order" and clicked the down green arrow to move it below "Image". Under this tab you can also add some custom text you would like displayed (I have not put any), but the "Custom Text" will have to be selected in the "Field Order" section.



Goodreads Sync: This plugin was developed by Grant Drake. Click here to go to the mobileread forum on this plugin. Some of you may have Goodreads accounts and you may want to synchronise your goodreads shelves with your calibre collection. This plugin helps with doing that. The figure below shows the icon of this plugin in the main tool bar along with available options.




The "Add to shelf" option allows you to add slected calibre ebooks to your "read", "currently-reading" and "to-read" shelves in your goodreads account. When I select the ebook "A Collection of Antiques" in my ebook collection and click add to shelf the following window appears.



Now just click "Add to shelf" to have it added to your appropriate goodreads shelf.

To use the second option "Sync from shelf", you need to first set it up. First click on the "Customize plugin" option. In the new window that opens, double click on an entry of interest under "Sync Actions" and  tell calibre what it should do with matching linked books. Now the synchronizing is set up and ready to use. Clkick on the "Sync from shelf" option under the goodreads icon. You may have some books on your goodreads shelves that you read paperback versions of and therefore do not have them in your calibre library. These books will be labelled as "No Calibre book linked" in the new window that opens. You can right click on the book and it will give you the option of adding an empty book to calibre with the metadata stored for your records. See figure below where I have added empty ebooks for the first two unlinked books.




Search the internet plugin: This plugin was developed by Grant Drake. Click here to go to the mobileread forum on this plugin.This plugin allows you to search the internetplugin in the main toolbar as well as the search options it allows for.



Some other plugins of interest may be Kindle Collections, Count pages and many more that are discussed here.

Hope you found this post useful. See you again in about a week.

Friday, November 11, 2011

calibre plugins: duplicate finder

The next few posts will deal with some of the more useful plugins in calibre. Functionality can be added to calibre via little code snippets called "plugins". The Calibre application itself comes with and makes extensive use of plugins developed by the Calibre development team. The Calibre plugins programming interface (API) makes it possible for users to write their own plugins, that offer additional features they find useful or override the default Calibre behaviour. For a full list of available calibre plugins click here. There is a help forum on mobile read dedicated to plugins. Please post here if you need help with developing or using a plugin.

Today we will discuss one popular plugin; "Find Duplicates". The goal of this plugin is to find duplicate entries in your collection so you can delete or merge them. This plugin was developed by Grant Drake. The help forum dedicated to this particular plugin can be found here.

Getting the Find Duplicates plugin: In the main calibre window click on Preferences and at the bottom left corner of the new window that opens click "Plugins". Now a window opens up with the list of available plugins. The "Find Duplicates" plugin is not on the default list so you will have to first get it. To do this, click on the "Get new plugins" button at the bottom left corner. A window with a list of plugins shows up. Choose the Find Duplicates plugin and click the install button on the bottom right corner. Restart calibre to see the "Find Duplicates" button in the main calibre tool bar.

Now click on "Find Duplicates" and you will see the following window:




Duplicates by author or title: The first choice in the duplicate search type as shown in the figure above is Title/Author. This allows you to locate duplicates by either author name or by title or both. You can set one column to ignore if you want only to use the other. The figure below shows the duplicates obtained in my collection by setting the "Title Matching" column to identical and the "Author Matching" column to ignore. I have also selected "Show all groups at once with highlighting".

The above figure shows all sets of books of the same title irrespective of the author names. Books of the same title are highlighted by the same color and grouped together. The two duplicate entries for say "The Columbus Dispatch" (highlighted in green) can be combined by selecting them both, clicking the little arrow next to "Edit Metadata" and selecting "Merge book records". Here you can choose to delete the extra files or keep them. Having done this, in the above figure, we are left with only one set of duplicates. 

Similar: The figure below shows the effect of setting the "Title Matching" column to identical and the "Author Matching" column to similar. Similar authors differ only in the punctuation or order of their names.
Soundex: The figure below shows the effect of setting the "Title Matching" column to identical and the "Author Matching" column to soundex. Soundex authors also differ only in the punctuation or order of their names but can also include books where the author names have minor spelling errors so they sound similar like "Austen" and "Austin" in the example below.



Fuzzy: The figure below shows the effect of etting the "Title Matching" column to identical and the "Author Matching" column to fuzzy. Fuzzy author matches compare using their surnames and only the first initial so they detect "Jane Austen" as well as "J. Austen" as shown below. This is the most general setting for finding duplicates.





Duplicates by ISBN or binary:
You can look for duplicates by comparing ISBN. This will allow you to search for books with identical ISBN and the author and title matching becomes disabled. Similarly Binary searching allows you to look for duplicate occurances of a file even if the title and author are different provided the actual files are identical. As shown in figure below "P&P" by "Unknown" contains the same file as "Pride and Prejudice" by "Austen Jane", "Jane Austin" and "J. Austen". They all contain the same identical text (.txt) file. While "Pride and Prejudice" by "Jane Austen" contains a different epub file.



Hope you found this post useful. See you in about a week with details on some more plugins.

Friday, November 4, 2011

apology: no blog post this week

Dear Blog Readers,

I am sorry but Kovid and I moved into our own apartment this week in India, so we have been very busy setting it up. I regret to say there will be no blog post this week. I will try my best to be back on schedule next week. Also please bear with delays in response on various help forums, bug reports and some what irregular releases. We will try to be back on schedule for all of these as soon as possible.

Thank you for your patience.

Sunday, October 30, 2011

calibre resources

calibre is a free and open source community driven project. The software has many layers. The most commonly used functions like format conversion, metadata editing, news fetching etc have big clear icons and are very easy to use. Yet calibre is very powerful and offers a lot of flexibility by way of customization options through "Preferences". Some customization options were discussed in earlier blog posts and a lot more will be discussed in the future. For the more advanced users who know how to use regular expressions, create custom style sheets, and/or have some minimum knowledge of python, calibre offers even more flexibility. So calibre, although accessible to anyone for basic functionality, is a very versatile and powerful program.

For those who want to learn to exploit the more advanced or subtle features of calibre, a lot of help is available. I hope this article makes you aware of various resources associated with calibre so you can make the most of it.

Mobileread: calibre was originally developed with the support of the mobileread community. There is a dedicated calibre forum on this site.  Kovid himself as well as a lot of long time calibre users and developers are members of this forum. The members of this forum are very friendly and responsive to help requests and questions. Many of them have a lot of experience with using the various subtle features of calibre and some are technically proficient as well. The best way to contact Kovid is to post your questions here. Also if your questions are detailed or technical in nature, this is the best place to get help.


The calibre mobileread forum was divided into a number of sub forums a few months ago. The sub forums are (see figure below):





  • Recipes: Here you can post requests for news sources. If you have made a news recipe for some source that is not already included in calibre we would appreciate it if you shared it here. Little contributions from a lot of people is what makes calibre such a versatile software. You should put your name in the author name section of the recipe for credit and it has to be licensed GPLv3. You can also post improvements to existing recipes here. If some existing recipe has stopped working make a post about it and the author of the recipe or somebody who has time will look into it and fix it as soon as possible.
  • Devices: This sub forum deals with device specific issues. You can ask for support for new devices here. So if your question or problem concerns a particular device like the Kindle or iPhone, this is the place to post it. Those of you who use apple devices may want to check out this sticky.
  • Catalogs: If you have questions about creating a catalog or list of ebooks in your calibre library, this is the place to post. calibre allows you to format your catalog and make it suitable to be printed as well as read on e-readers. There are a number of settings that you can find out about here.
  • Conversion: calibre's conversion engine is sophisticated and has many features you may not have encountered yet. This is the place to get help on ways to customize the conversion process.
  • Library Management: calibre allows you to play around with the ebooks metadata and present it in various ways. Any questions about downloading metadata, editing it, or exporting it can be asked here. You can also ask about the tag browser and managing and viewing your ebooks.
  • Plugins: A lot of calibre's features are implemented by way of plugins. These are for less commonly used features that may be useful to specific subsections of people. This system makes calibre versatile, yet simple to use. This sub forum is for requesting new plugins or for getting help with existing plugins.
  • Development: This is for those of you who want to work on calibre and add to its capabilities. You can get help from Kovid and other developers to smooth the way for you. calibre relies on inputs from various developers who have made small and large code contributions over the years. Some of them have found working on calibre to be a fulfilling experience. One thing to remember is that although you have full credit for your code  patch all code submitted to calibre must be licensed GPLv3 and cannot be proprietary.


The screen names of the calibre mobileread moderators are Starson17, kovidgoyal, Piper_, GRiker, theducks, Manichean, kiwidude, ldolse, dwanthny, chaley, user_none. Many of them have made code and recipe contributions to calibre in addition to the help they provide by way of answering questions.

Mobileread also has a number of other forums for various book devices and ebooks and authors that you may find useful.

Facebook: For those of you with facebook accounts, calibre has a facebook fanpage which we use as a help forum. You can post any questions concerns or feedback you have for calibre. It will be addressed there or you will be directed to a more appropriate place for help. The facebook fanpage is managed by Krittika Goyal (me). I answer most of the questions so please be patient with me because sometimes I may be busy and wont get to a question for a few days. Also I may not know the answer to some questions and it takes me a while to find out.



Twitter: For those of you who tweet, calibre has a twitter page called "calibreforum", run by Kovid's father Niraj Goyal. However if you have a detailed question opt for one of the other help forums because the 140 character limit on twitter makes answering difficult.

Help forum etiquette: The volunteers in the various calibre help forums are very friendly. They are happy to help with questions of all kinds from the trivial to the very technical. However please be polite in your questions and patient with those helping you. This is the quickest way to get your questions answered. Please do not vent your frustrations on those who are volunteering their time to help you.

Self help: calibre has been well documented both for the benifit of it's developers as well as it's users. There is an extensive FAQ as well as detailed user manual. When you encounter a problem first check out the FAQ. If your question has been adressed in the FAQ you have a quick solution and you will save some time for the volunteers at the help forums.  There are also a number of helpful demos and video tutorials to be found here. With time we hope this blog will also contain useful articles that you can use as a reference.

Open Books: calibre is an open source software and we firmly believe in the open source philosophy. Digital Rights Management (DRM), in addition to being a source of inconvenience to users, is one of the roadblocks to exploiting all the features of calibre. While we don't believe in breaking the law, we do believe it needs to be changed. In our efforts to eradicate DRM, we have started "Open Books", a catalog of DRM free ebooks. The idea is to give publicity to DRM free ebooks and their authors as well as to provide calibre users with a large list of DRM free ebooks to choose from so they can truly use calibre to it's full potential without breaking any laws.

Open Books, a site for easy browsing of DRM-free e-books that are not in the public domain. Most public domain ebooks are available DRM free at the Project Gutenberg site. Open Books is a compilation DRM free e-books from various sources linked to enable readers to browse and download them.
Open Books now lists over 2730 books from over 30 stores and features over 1020 authors.






I hope you will make use of these resources to improve your calibre experience and some day become an active member of the calibre community contributing help at one of these forums. This article has been a digression from our usual format of tips and tricks but I hope you find it useful. Next week we will be back with a more usual article. Have a good week!



Friday, October 21, 2011

calibre's tag browser

The calibre tag browser is a versatile and powerful tool with some obvious and many obscure but extremely useful features. Today I will discuss some of these features. The figure on the left below shows the part of the main calibre window which is the "tag browser". One of the more obscure features I will discuss is how to make subgroups in the tag browser as shown in the portion highlighted in blue on the right figure below.



Show ebooks with tag: To explore a particular category in the tag browser, for example "Tags", click the little arrow next to it (Tags) and a list of all the tags will appear. Suppose you are interested in ebooks tagged "Romance" then click on it and a green "+" sign appears next to it indicating that the ebooks  now displayed in the main calibre window are tagged "Romance". The tag browser allows you to choose multiple tags simultaneously. For example, while the "Romance" tag has been clicked, if you hold down the "Ctrl" key on your keyboard and select another tag, say "Family Life" then the green "+" sign appears next to both the tags as shown n the figure below. Now at the bottom right corner of the window in the second entry if you see "Match any", then the main calibre window will display all ebooks that have the tag "Romance" as well as all ebooks that have the tag "Family Life" as shown in the figure below. However if you click on "Match any" a menu shows up with another option; namely "Match all". If you select "Match all" then the main calibre window will display only those ebooks that have both tags and unlike the figure below only "Pride and Prejudice by Jane Austen", which has both tags "Romance" and "Family Life" will be displayed. Here you could have chosen the two tags from different categories for example "Humour" (from Tags) and "George Bernard Shaw" (from Authors) and "Match all" to get all ebooks tagged "Humour" by George Bernard Shaw or "Match any" to get all ebooks tagged "Humour" as well as all ebooks by George Bernard Shaw.


Show all ebooks w/o tag: Now instead of displaying all ebooks with a particular tag, you may want to display all ebooks that do not have a particular tag. For example, if you want to see all your ebooks except the ones written by George Bernard Shaw, click the little arrow next to the "Authors" category to get the list of authors and then click on George Bernard Shaw. A little green "+" sign appears next to it. Now click again on George Bernard Shaw. This time a little red "-" sign appears next to it indicating that the ebooks displayed in the main calibre library include all your ebooks except those by George Bernard Shaw. By holding down the "Ctrl" key you can select multiple entries. After having eliminated all ebooks by George Bernard Shaw if you hold down the "Ctrl" key and left click twice on "calibre" a red "-" sign appears next to "calibre" as well as shown in the figure below and in the list of ebooks displayed in your main calibre window you will see your entire ebook collection except those authored by either George Bernard Shaw or calibre.


Sorting tags: Calibre allows you to sort the tags in the tag browser in many different ways. For example the list of authors in the above figure are displayed in alphabetical order of their last names. This is because, the first entry in the bottom left corner is set to "Sort by name". However if you click on it it gives you a menu with a number of options. If you select the option "Sort by popularity" instead, then the authors will be displayed in descending order of the number of ebooks by them in your library. You can also choose "Sort by rating" in which case the authors will appear in decreasing order of their average rating (by stars assigned to their ebooks in the ratings column).


Making subgroups of tags: The top figure on the right shows a subgroup of tags to the main tag "Classics". To create such subgroups you need to enable this feature. To do this, go to Preferences -> Look and Feel and click on the "Tag Browser" tab. Here in the field for "Categories with hierarchical items" enter "tags" and click "Apply" in the top left hand corner. Now to actually get hierarchical tags, in the tags section of the metadata entry enter a tag of the form MainTag.SubTag. Like in the top figure on the right and example would be Classics.Russian or Classics.English.

Managing categories and tags: If you right click on a particular category in the tag browser a menu appears. The first option allows you to hide the category so you can reduce clutter if you are not interested in it. The fourth option allows you to manage and create user categories. Say, two people John and Joe use the same calibre library. But Joe would like to maintain a separate list of his tags so he only has to search through his own. To do this, right click on a category in the tag browser, choose the "Manage User Categories" option, a new window opens as shown in the left figure below. Enter the title say "Joe's tags" in the right hand top corner and click the green "+" sign. Now you have the new user category called "Joe's tags" that shows up in the left hand top corner as in the figure below. You can now select the tags on the left and use the blue arrow to copy those tags into the "Joe's tags" category. When you are finished with copying the tags click "OK" and now the new category with it's tags appear in the tag browser as shown in the right figure below. Now if you right click on this user category, you willsee an option that allows you to create sub categories. In addition if you right click on any tag in the tag browser, you will see the option to manage tags, that allows you to rename and delete tags.


Hope you found this post useful. See you in about a week.

Friday, October 14, 2011

ebook format conversion

A lot of you have used calibre to convert your ebooks between different formats. The calibre conversion system is very sophisticated and has many subtleties. Today I will discuss just a few features that are often overlooked and yet a lot of you may find very useful. The following is the "Convert books" icon in the top main calibre tool bar:



Bulk convert:
Calibre allows you to convert books in bulk. You can select a set of books, all of which may or may not be in the same format, and convert them all in bulk to a different format. So say you have a kindle and you have a number of books in EPUB, RTF and PDF formats. You can select them all. Then click the "Convert books" icon or click the little arrow next to the "Convert books" icon and choose the option "Bulk convert". A new window  opens. At the top right corner of the window you can set the output format, which in this case would be MOBI.

calibre allows you to choose various settings for the conversion output. A few of these are discussed below. When you select a book and click the "Convert books" icon  in the top tool bar a new window opens. On the left of the window is a menu with a number of entries.

Smart Punctuation: The second entry in the menu is "Look and Feel". When you select it on the right you will see a number of options to customize the look and feel of the output of the conversion. Under the section called "Text justification" there is an entry called "Smarten punctuation". When this is selected before the conversion, all plain quotes, dashes and ellipsis are converted to their typographically correct equivalents. For example, plain quotes to curly quotes. You can do the reverse by clicking "Unsmarten punctuation"

Page Setup: The fourth entry in the menu is "Page Setup". This option allows you to optimize the conversion output for a particular device, specifically the image sizes so they fit well on the screen of the device. So say you have a Kindle, then you would choose "Kindle" in the list of "Output Profiles" that appear when "Page Setup" is selected. Say you have converted all your ebooks for your Kindle and stored them that way, and now you have bought an iPad. To optimize the output for ebooks converted to EPUB for the iPad from MOBI for the Kindle, select "Kindle" in the list of "Input Profiles" and "iPad" in the list of "Output Profiles".


Covers and Metadata: The fifth entry in the menu is "Structure Detection". When you select it on the right side you will see the option called "Insert metadata as page at start of book". Selecting it will include the metadata at the beginning of your ebook itself, including reviews if they are available. Suppose you do not want to convert the ebook to a different format but want to include the metadata and cover within the ebook, then just select the same formats for the input and output during conversion with the "Insert metadata as page at start of book" option checked. See figure below for an example. The metadata along with a choice of covers for an ebook can be obtained by selecting the ebook and clicking the "Edit metadata" button in the top main calibre tool bar.


Some ebooks have an image as a cover which is not explicitly marked as the cover. Instead it is just another image in the ebook. In this case when you convert the book in calibre, the output may have two covers. If this happens, re-do the conversion with the "Remove first image" option (above the "Insert metadata as page at start of book") selected. This will give you an ebook with just one cover.

MOBI Table of Contents: For the specific case of converting to MOBI, the eighth option on the left menu is "MOBI output". For MOBI output calibre generates a Table of contents (TOC), which is inserted by default at the end of the ebook. When the "MOBI output" option is selected in the left menu you can choose "Do not add Table of Contents to book" if you do not want the TOC or choose "Put generated Table of Contents at start of book instead of end" if you want the TOC at the beginning of the book. The TOC calibre generates is hyperlinked to the chapters.

Saturday, October 8, 2011

Custom news fetching

The "Fetch News" feature in calibre is very powerful and versatile. It has over a 1000 built in news recipes, spanning over 30 languages and 50 countries, for websites of newspapers, magazines and blogs. At the click of a few buttons you can set up regular downloads of the news source of your choice. Not only can you choose from this large variety of available recipes, but you can also, create a customized recipe of your own or tweak one of the available recipes to make it better suited to you. In the past creating pretty news recipes has been a job for those with some expertise in python. However, thanks to the "auto clean up" feature recently included in calibre, for many a website or combination of websites, anyone can follow a simple formula to write a recipe. This may not work for a small fraction of complicated websites but is well worth a try. The "Fetch News" icon in the top calibre toolbar is shown in the figure below.







Basic news fetching: To get started just click on the above icon in the top toolbar in the main calibre window. A new window, shown in Figure: 1 below, opens up. It lists all available news sources by language and country. For example "Arabic[2]" on the second line indicates there are 2 news souces available in the Arabic language. To see what these news sources are, click the little arrow on the left of "Arabic". You will notice in case of English there are a number of entries. This is because a number of different countries publish English newspapers, magazines and blogs. The list indicates there are 7 news sources from australia, 1 from Bulgaria, 23 from Canada and so on.




If you are looking for a particular news source, say the "Washington Post", then you can search for it as shown in Figure: 2 above. Type in all or some the letters in the search bar and press enter the list below will reduce to include only matches to your search, so as shown in Figure: 2 above only 3 news sources match up to the word "washington". Now if you select "The Washington Post", options appear on the right side that allow you to download the news immediately (bottom right, "Download Now" button), or to set up a schedule for automatic downloads. The automatic download will occur the first time calibre is run on your computer after the scheduled download time.

Adding RSS feeds to existing news recipes: There are some very simple things you can do to customize the news recipes to your taste.
News recipes contain different RSS feeds for different sections like Sports, Politics, Business etc. You may be interested in a feed that is not included like say Entertainment.
It is very simple to add this feed of interest to you.

First go to the news website of interest. Search the page for RSS (RSS feeds) indicated usually by a little rectangular orange icon. Now click on this link and it should take you to a page with RSS links to various sections of the news like Sports, Politics Business etc.

Startup calibre and click the little arrow on the fetch news and click on "Add a custom news source". A new window opens up and on the bottom left corner click on "Customize builtin recipe". Now a little window opens up where you can pick the recipe of the news scource you wish to customize.

Example : The Los Angeles Times

Now on the left column of the previous window The Los Angeles Times should show up. Select it and the recipe will show up on the right column.

If you look at the recipe you will find a block with RSS feeds that begins with

    "feeds = [ ...
             


Instead suppose it had looked like (with fewer feeds and not including the feed on the Sports which say you are intersted in)

    feeds = [
              (u'Top News'             , u'http://feeds.latimes.com/latimes/news'                           )
             ,(u'Local News'           , u'http://feeds.latimes.com/latimes/news/local'                     )
             ,(u'National'             , u'http://feeds.latimes.com/latimes/news/nationworld/nation'        )
             ,(u'National Politics'    , u'http://feeds.latimes.com/latimes/news/politics/'                 )
             ,(u'Business'             , u'http://feeds.latimes.com/latimes/business'                       )
             ,(u'Education'            , u'http://feeds.latimes.com/latimes/news/education'                 )
             ,(u'Environment'          , u'http://feeds.latimes.com/latimes/news/science/environment'       )
             ,(u'Religion'             , u'http://feeds.latimes.com/latimes/features/religion'              )
             ,(u'Science'              , u'http://feeds.latimes.com/latimes/news/science'                   )
             ,(u'Technology'           , u'http://feeds.latimes.com/latimes/technology'                     )
             ,(u'Africa'               , u'http://feeds.latimes.com/latimes/africa'                         )
            ]


Then all you have to do is follow the previous syntax and add in the name and get the link from the page of the website of The Los Angeles Times with the RSS feeds corresponding to Sports.

Make sure the link you get looks similar to the other links in the recipe. If not try to copy the link from the little square orange icon.

If there are RSS feeds corresponding to sections that do not interest you, you can delete the names and links corresponding to those sections. This will make the download process faster an remove clutter.

Then click the Add/Update recipe button at the bottom left corner. Now a new "replace recipe?" window opens up. Click replace recipe and you are done!

To access this recipe go to the main calibre window and click "Fetch News" and you get a list of news sources. The first entry is Custom. Click on it and it will expand to show the list of your customized news sources.

Auto clean up: This is a powerful feature that enables lay users to make custom recipes. You may be interested in making a single news recipe that has RSS feeds from different blogs and news sources that you visit. This can be done quite easily with "Auto clean up". The following recipe obtains the RSS feeds for the politics section of 3 different news sources, namely, "The Seattle Times", "The San Francisco Chronicle" and "The Los Angeles Times":

from calibre.web.feeds.news import BasicNewsRecipe


class Politics(BasicNewsRecipe):
    title          = u'Politics'
    language       = 'en'
    __author__     = 'Krittika Goyal'
    oldest_article = 3 #days
    max_articles_per_feed = 20
    use_embedded_content = False

    no_stylesheets = True
    auto_cleanup = True
    auto_cleanup_keep = '//div[@class="thumbnail"]'


    feeds          = [
('Seattle Times',
 'http://seattletimes.nwsource.com/rss/politics.xml'),
('San Francisco Chronicle',
 'http://feeds.sfgate.com/sfgate/rss/feeds/blogs/sfgate/nov05election/index_rss2'),
('LA tIMES',
 'http://feeds.latimes.com/latimes/news/politics/'),
]

The first line in red must be in every news recipe. The next block of code in grey is information like title author etc, which you should change to suit your recipe.  The next two lines in red are what is used to clean up the web page, remove advertisements and other unwanted material. The "auto_cleanup" uses statistical analysis to extract the useful content in the news website or blog. I will return to the blue line later. The next grey block of code includes the feeds of interest. The output you get (w/o the blue line) for one of the pages of "The San Francisco Chronicle" is shown in the figure below.



"auto_cleanup" is usually great at picking out the relevant content from a variety of websites, so you do not need to manually clean up each website. As a result you can use feeds from different websites even if the articles have very different structures. Sometimes however, "auto_cleanup" can be over zealous and remove content that is indeed relevant like a picture. To fix this you need to understand a little bit of HTML. You need to use "firebug" in firefox or a similar tool to find out the tags corresponding to the part that you would like to remain.

In the above example "auto_cleanup" was removing the picture at the beginning of the articles in "The Los Angeles Times".  To fix that we had to add the blue line of code. Now the picture is included (see figure below).



Without the blue line of code, you would still get the text of the article but the picture would be missing. The "auto_cleanup" feature is based on code from the ReadItLater open source project.


For the more advanced user: Finally for those of you who are adventurous or experienced in programming, the customizing news feature in calibre is very powerful and for tips on using it to its full potential visit the "tips for developing new recipes" section of the calibre user manual.

The great thing about calibre is that its features are accessible at many levels so both lay users as well as advanced tinkerers find it useful and enjoyable.

Finally sorry for this blog post being later than usual, but Kovid and I moved to India today and the moving procedure kept me very busy. We still need to settle in so the next few blog posts may be a little off schedule as well and i may e a little slow in responding to comments. I will do my best to be on time. As always see you in about a week and hope you found this post useful.

Friday, September 30, 2011

all about "Get Books"

"Get Books" allows you to search for ebooks online, that you can then
purchase/download to your calibre library. It can simultaneously search over 40 different online stores for books by title and author. The returned list of matches can be sorted by price, allowing you to find the cheapest edition of the book. To access "Get Books", click the icon shown below in the calibre tool bar.




Parallel Searching:
In the latest version of calibre (0.8.21 right now) "Get Books" searches over 40 stores from various countries for ebooks. These stores include ebooks in English, Spanish, German, Russian, Polish, Dutch, French and Italian. The search results only show ebooks and exclude paperbacks, hardbacks etc. However you do not have to search for the ebooks in all the stores. You can choose the stores to be searched by selecting them from the list on the left. In the figure below  as you can see on the left column 3 stores have been selected so "Get Books" searches on those stores.





You can search for a particular ebook by entering the title in the search box or you can search for all ebooks by a particular author by entering the author name in the search box. In the figure above the search has been performed for the title "the naughtiest girl in school". One could enter "Enid Blyton" in the search box to see all ebooks by Enid Blyton available in the selected stores. It is also possible to simultaneously search for an author and title as shown in the figure below (title is "Youth" and author is "Asimov").



Price comparison: It is interesting that the price of the ebook in the
different store varies quite a bit as shown in both figures above. "Get books" allows you to do price comparisons over a very large range of stores when buying ebooks. Project Gutenberg, Archive.org, mobileread.com and Manybooks, between them, carry almost all the public domain ebooks and make them available free. So if you are looking for a public domain ebook like "Youth by Asimov", one of these may be your best option.

DRM status: Price may not be the only thing that determines where you buy your ebook from. "Get Books" also tells you the DRM status of the ebook. A "red lock" icon in the DRM column implies the ebook has DRM. This means it can only be read on particular devices so be careful to check compatibility with your device. A "green open lock" implies the ebook does NOT have DRM so you can get this ebook for any device. Even if the ebook is not available in the format you would like, you can convert it to a format of your choice using calibre. A
"blue question mark" implies the DRM status is not specified by the store and therefore not known. In the figure above Amazon has two versions of the ebook, one is DRM-free and costs $1.49 and the other has DRM and costs $0.99. Barnes and Nobles also has two version of the ebook both for $0.99 but the DRM status on both are unknown. The Project Gutenberg and Feedbooks versions on the other hand are both free and DRM-free.

For more information on DRM visit "about DRM".

Language tip: Typically if you're looking for a book in a specific language then searching for the title in that language is the best way to go about finding it. For example searching for "Adventures of Huckleberry Finn" only gives results in English but searching for "Las aventuras de Huckleberry Finn" gives results in Spanish.

Internal browser: As shown in the first figure, if the little box marked "Open in external browser" is not checked then when an entry in the list is clicked on it will take you to the sale or download page in calibre's internal browser. the advantage of using the internal browser is that if you are getting your ebook from Archive.org, Feedbooks, Manybooks or Project Gutenberg,the books downloaded directly from the "Get Books" search dialog into your calibre library; otherwise for every store, except Amazon, the internal browser detects ebook downloads and automatically adds them to the your calibre library. Books from Amazon automatically open in an external
browser.


Affiliate programs: By using "Get Books" you could help support various calibre developers at no cost to you. The various "Get Books" stores were set up by different calibre developers. Some of them have affiliate programs and there are marked with a little red heart as shown in the rightmost column of both figures above. When you buy an ebook from one of these stores with the affiliate program, the developer who set up that store gets a small percentage of your purchase through the stores affiliate program and this is at no
additional cost to you. If you want to know the name of the developer you are supporting just hover your mouse on the heart icon and a little information bubble will tell you the name of the developer. Most of the American stores are courtesy of John Schember while most of the European stores are courtesy of Charles Haley.


Something to keep in mind:
"Get Books" aggregates stores from all over the world and while it may present results this doesn't necessarily mean the store will sell to you. Many stores (pretty much every one employing DRM) have geographical restrictions on who they can/will sell to. Get Books only tells you who has it and for what price; it doesn't tell you if the store is willing to take your money for the book.

I would like to thank calibre developer John Schember for his help with this article and for the instrumental part he has played in the existence of "Get Books".

Friday, September 23, 2011

the calibre viewer

Today we explore the many features of calibres ebook viewer. calibre's ebook viewer allows you to view ebooks in all of the 22 input formats that calibre supports conversion from. These include CBZ, CBR, CBC, CHM, EPUB, FB2, HTML, HTMLZ, LIT, LRF, MOBI, ODT, PDF, PRC, PDB, PML, RB, RTF, SNB, TCR, TXT, TXTZ, and no, I did not make that up ;-) They are all existing ebook formats.


However by default only files in EPUB, FB2, HTML, LIT, LRF, MOBI, PDB, PRC nad RB formats are opened in the calibre internal viewer. This is because for the remaining formats, usually a better result is obtained by viewing in an external viewer; like acroread or okular for PDF. For these other formats calibre will automatically open the files in an external viewer unless you ask it to open in the internal viewer. To choose which formats you would like to view using calibre's internal viewer go to Preferences -> Behavior -> and select the formats you want in the right column.


The buttons

The top left corner of the viewer gives you a location as a decimal number. This does not change when you change the viewer window size or the font size. So you can always use that number to refer back to a particular part of the book. The buttons in the ebook viewer have been labelled in the figure below. We will go through the functionality of each.




Back and forward buttons: These buttons will only be functional if your ebook has links in it that take you to other parts of the ebook or if you click on an entry in the table of contents (discussed next). Once you click on a link you can use the back button to take to the place you were before you clicked the link. These buttons are similar to back and forward buttons on a browser.

Table of Contents (TOC): Clicking this button opens up a TOC as shown in the figure below. You can navigate through the chapters of the book by clicking on the entries of the TOC. The figure shows that when the entry "ABSENT TREATMENT" is clicked in the TOC the viewer takes you to that chapter. Clicking on the TOC button again will make the TOC disappear.



Previous and Next page: These buttons should be used to navigate pages. Not the back and forward buttons which become activated only when a link is clicked or the TOC is used.

Bookmark: When this button is clicked a window opens that allows you to choose a helpful name for the bookmark so you can refer to it easily later.

Reference mode:
When this button is clicked and the mouse is hovered over any text, a unique number referring to the paragraph to which the text belongs is displayed. This number is independent of viewer window size or font size and can be used to refer to the paragraph.

Preferences: Clicking on this button opens a new window with three tabs. The first tab is "General" and here you can choose the default font you would like in each font family as well as the font family. You can also choose some window and mouse settings to suit your convenience.
The next tab is "Keyboard Shortcuts". The calibre has a default set of shortcuts for taking you to different sections of the ebook or for scrolling. For example by default the viewer uses the "J" or "Down arrow" key to scroll down the document, however if you dont like this choice you can double click on it. Initially "Default" is selected, but you can select "Custom" and it will let you choose upto 2 keyboard shortcuts.

The third tab is "User Style Sheet". Here you can customize the background colour, indentation, relative font sizes of section titles or chapter names etc. To do this you have to create a custom style sheet (CSS).  Here is an example:

body {color:black !important;
background-color:rgb(228,200,200) !important;
text-align:justify !important;
text-indent:3em !important;}
h1, h2, h3, h4, h5, h6
{color:red !important;
text-align:center !important;
font-style:italic !important;
font-weight:bold !important;}

The first line sets the text color to black. On the second line is a command for the background color in RGB. The particular values chosen here generate the pink background in the following figure. The third line ensures the text ends at the same position on the right end of the viewer (compare figure below with figure above). The fourth line specifies the indentation at the beginning of each paragraph. The lower set of commands sets the text style for headings (hi, h2 ... , h6) to have color red, aligned center, in italics and bold. So even if you don't know how to create a CSS all on your own just copy an example CSS and tweak the values to get the colors alignment and indentation you want. Notice the "!important" in every line of the CSS. That is required to override the style already specified in the epub book. To see the effect of each of these elements of the style sheet compare the figure below which uses the above CSS with the one above (no CSS). The TOC was removed in the figure below by just clicking on the TOC button.




CSS allows you to make many other powerful stylistic changes. For help with writing CSS or for some more examples visit the CSS thread on calibre's library management forum on mobileread.




Metadata: Clicking on this button displays the metadata of the book on the viewer screen. See figure below


Hope you found this useful. Please feel free to comment. See you in about a week.