What is Waartaa?

Waartaa or wārtā is a word in Hindi: वार्ता, which means to communicate. And that's what waartaa is for. Waartaa is a web based IRC client as a service and it facilitates centralized logging, idling functionality, unique identification across multiple clients and a rich UI for awesome user experience. Waartaa is open sourced under MIT License. The source is at https://github.com/waartaa/waartaa/ .You can download, fork, customize and setup Waartaa as a service anywhere, be it a single user laptop/desktop, server for your self and your friends.

galleries/waartaa/waartaa.png

Why?

There are an arsenal of IRC clients and tools, so why another one?

Waartaa is not just a random fun project, although it has been fun and full of adventures to work on it. Below are a few reasons why I started to work on Waartaa:

  • GUI IRC clients work only for single machines. It's a pain to sync logs across multiple devices across multiple IRC clients.
  • Local IRC clients do not let you idle when you are not online. It's not possible for everyone to get a server to setup ZNC or similar idling server.
  • Most desktop IRC clients do not have that WOW! look and feel.
  • Lack of identity when logged in from multiple devices simultaneously.
  • Local network configuration (proxy, firewall) and quality (speed, timeout) often becomes a hindrance to good IRC experience.

Waartaa solves the above issues as follows:

  • Running IRC client as a service on better infrastructure ensures that you are always connected to IRC and capturing IRC logs.
  • Waartaa serves as a central place to store your chat logs.
  • No matter what device you login from to Waartaa, YOU are always YOU in the IRC and not some YOU_, YOU__, etc.
  • Waartaa is built on top of web technologies. So, it works flawlessly and uniformly across multiple platforms and looks equally awesome in all the them. This adds up to a superior user experience.

Features

Beautiful and useful chat interface

galleries/waartaa/waartaa_chat_logs.png
galleries/waartaa/waartaa_nick_options.png

Easy to join server/channel

galleries/waartaa/waartaa_add_server.png
galleries/waartaa/waartaa_channel_join.png

Stylish menus

galleries/waartaa/waartaa_server_menu.png
galleries/waartaa/waartaa_channel_menu.png
galleries/waartaa/waartaa_channel_nick_menu.png

Informative

galleries/waartaa/waartaa_channel_connecting.png
galleries/waartaa/waartaa_channel_unread_msg_count.png

Under the hood

  1. Meteor JS http://www.meteor.com/
  2. MongoDB
  3. Forked node-irc https://github.com/waartaa/node-irc
  4. And a host of meteorite apps from https://atmosphere.meteor.com/

How can I contribute?

Comments



galleries/MozSummit2013/summit_logo.png

I have been contributing to Mozilla for some time now and so, I got the opportunity to attend Mozilla Summit 2013 at Santa Clara. Contributors from all around the globe turned up to attend the Summit which was going in parallel in 3 places: Santa Clara, Brussels and Toronto.

galleries/MozSummit2013/moz_summit_group_pic.jpg
Finally, I met Mathjazz at the Summit. We had been working for a long time on Pontoon and were communicating over the internet. Not only did I meet Mathjazz, but his Mozilla team from Slovenia. We had some discussions on Pontoon and it's roadmap. I attended the i18n and l10n discussions led by Axel from Mozilla and got more insight into the process. I had some discussion with Axel on plans to integrate l20n with Django and Python in general. It did not seem that straight forward. Axel asked me to get in touch with @jezdez on this.

I also met the Erik Rose, the creator of nose, dxr and many other awesome tools. I had contributed to Erik's nose-progressive in the past. It was an honour to have a conversation with him.

galleries/MozSummit2013/erikflo.jpg
3 days passed like anything. There were loads of awesome talks and discussions and group activity. I was involved in brainstorming on what Mozilla could do in the Cloud space along with Sayan, erickt and others. It was great to meet fellow Mozillians from around the globe. I enjoyed the chats I had here and there with fellow Mozillians during the conference.
galleries/MozSummit2013/dgplug_folks.jpg
As for me, I had been mostly busy hacking on Waartaa during the conference. Well, Waartaa is "A web IRC client written in Meteor JS. It is aimed towards being an intuitive, collaborative IRC client across multiple devices of the user along with centralized logging". Have a sneak peek into it.
galleries/MozSummit2013/waartaa.png
I tried to speak with as many people as I could about my project and I was looking for potential contributors. I also pitched about Waartaa on stage, but it did not go well. The open session on waartaa was worse. It was just me and 14 empty chairs. It felt terrible, but I kept hacking on it all by myself. I believe in the idea and that's what that keeps me driving. Fortunately, I met Marienz from Freenode and shared with him my work and idea on Waartaa. I expressed my concern with the privacy of user data when using and the trust factor. He gave me some useful pointers towards SASL, IRC extensions, etc. that'd help me to find a solution for the issues we discussed.
It was great to be part of Mozilla Summit. Events like this get you charged up to start contributing to Open Source in full throttle. After coming back from the event, I started contributing to Firefox's devtools. Till date, 3 of my patches have been merged to Firefox's codebase. It feels great. Also, I made substantial progress on Waartaa. Sayan started contributing to Waartaa. I hope to get an initial release very soon.

Comments



Well, it's been quite some time since I last blogged. There have been a lot of things happening at my end and amidst this I totally dropped the ball on updating my posts. As it is said, it's better late than never. Let me summarize what happened over the past couple of months.

Good bye Transifex, hello Goibibo

I quit #Transifex last January (2013) and joined #Goibibo. Goibibo is one of the products of Ibibo Web Pvt. Ltd., an e-commerce giant in India. No doubt Goibibo is doing an awesome job in the market, one of the main reasons for me joining Goibibo was that it is a #Python shop and #Django is what that drives it. Needless to mention that we Python lovers always favour Python and love to stick to it :D. So far, it's been an #awesome #roller-coaster ride at Goibibo, especially, in terms of the scaling it handles. Things are totally different when your product serves so many users and especially when you are dealing with monetary transactions. The things I learned at #Transifex helped me to push some best practices like TDD, coding standards (Flake8), etc. in Goibibo's work culture. I also replaced #gitweb + #reviewboard with #Gitlab as our VCS. Gitlab has helped a lot to streamline the process of pushing code. People now work on #branches for each feature/fix. The #branch gets tested by #QC and the #developers first. If everything is OK, the branch gets merged to master and pushed for a release. Currently, I am planning to implement #DXR codesearch and #IRC for communication.

Python Month 2013

This year, a new initiative was taken by Pycon India group. It was to conduct Python workshops all across India to spread Python awareness in the month of August 2013. I and Sayan have been regularly attending pre Pycon meetups (mostly in Goibibo's office) and decided to volunteer for the same. I and Sayan conducted two workshops on basic Python: one at BNMIT and the other at Reva Engineering College in Bangalore. We met a lot of inquisitive youngsters in our workshops. We did our best to motivate them and contribute to Open Source.

Pycon India 2013

This year's Pycon was super awesome for many reasons. We had a large gathering of #dgplug folks for the first time in any conference. We had loads of discussions, brainstorming on project ideas and some hacking. I conducted a workshop on "Test your code" on the first day of Pycon. I got the chance to meet the Kenneth Reitz, the creator of request. I was greatly inspired by the keynote speech by Kiran. Among other talks, I also found Anisha's talk on Functional testing with Python quite interesting. This year, Goibibo sponsored Python and we had a Goibibo stall at the venue. We had quite a lot of activities going on at our stall. It was great fun. I spent most of the time during the conference hacking on waartaa. I got some tips from Kaustav Das Modak on optimizing mongodb for waartaa. Most of the days ended with a #dgplug team dinner. The following day after Pycon, I, Sayan, Kushal and Kenneth went for a trip to Mysore. It was great fun.

Well, that'll be all for now. It's getting an overkill for a single post ;)

Comments



Finally, I have created my first portfolio cum blog website at www.rtnpro.com. I used Nikola for this.

The theme for the website uses the theme of Twitter bootstrap as it's base. A cool thing about my home page is that the projects section is dynamically generated using MyGithubProjects to show the projects that I work on in Github: projects those I own and upstream projects I contribute to. So, I need not maintain the list of projects any more for my portfolio ;)

It took me some time to integrate MyGithubProjects as task in Nikola to generate the home page from a custom template. Also, playing with styling for the page was fun. After putting all this time and effort, it seems pretty good enough to satiate me.

You can find the code for my website here.

Comments



Lately, I found out that Django's TransactionTestCase leaves test data in database after the test case is executed. It's not until the next execution of _pre_setup method of a TransactionTestCase instance that the database is flushed. This is troublesome when tests are run with Django Nose's test runner with REUSE_DB =1.

An easy fix to this is to customize the TransactionTestCase so that it deletes the test data on exit. I wrote a simple wrapper around Django's TransactionTestCase and extend it to write other transaction test cases.

from django.test import TransactionTestCase
from django.db import connections, DEFAULT_DB_ALIAS

def flushdb(cls):
    if getattr(cls, 'multi_db', False):
        databases = connections
    else:
        databases = [DEFAULT_DB_ALIAS]
    for db in databases:
        management.call_command('flush', verbosity=0,
        interactive=False, database=db)


class BaseTransactionTestCase(TransactionTestCase):
    @classmethod
    def tearDownClass(cls):
        flushdb(cls)

Comments



My proposed talk titled Develop for an international audience got selected for Pycon India, 2012. It's time to start working on the slides. I am thinking to use rst to write my slides. Also, I have booked by flight tickets for Pycon :)

Thanks everyone who voted for my talk.

Comments



Transifex already supported validating translations of old styled Python strings, e.g.,

"A sample string with a %(keyword)s argument." % {'keyword': 'key word'}

The validation is done by checking if all the positional and keyword arguments are present in the translation string and the translation string does not contain any extra argument which is not in the source string. You can have a look at the validator code here.

However, the existing validator is not able to check for replacement fields in new style Python format strings, e.g.

"This is a sample string with different replacement fields: {} {1} {foo["bar"]:^30}".format(

"arg0", "arg1", foo={"bar":"a kwarg"})

I tried to devise a regex to extract the replacement fields in the Python format string based on the grammar defined here.

# Regex to find format specifiers in a Python string

import re

field_name = '(?P<field_name>(?P<arg_name>\w+|\d+){0,1}'\
                '(?:(?P<attribute_name>\.\w+)|'\
                '(?P<element_index>\[(?:\d+|(?:[^\]]+))\]))*)'

conversion = '(?P<conversion>r|s)'
align = '(?:(?P<fill>[^}{]?)(?P<align>[<>^=]))'
sign = '(?P<sign>[\+\- ])'
width = '(?P<width>\d+)'
precision = '(?P<precision>\d+)'
type_ = '(?P<type_>[bcdeEfFgGnosxX%])'

format_spec = ''\
    '(?P<format_spec>'\
        '%(align)s{0,1}'\
        '%(sign)s{0,1}#?0?'\
        '%(width)s{0,1},?'\
        '(?:\.%(precision)s){0,1}'\
        '%(type)s{0,1}'\
    ')' % {
        'align': align,
        'sign': sign,
        'width': width,
        'precision': precision,
        'type': type_
}

replacement_field = ''\
    '\{'\
    '(?:'\
        '%(field_name)s{0,1}'\
        '(?:!%(conversion)s){0,1}'\
        '(?:\:%(format_spec)s){0,1}'\
    ')'\
    '\}' % {
        'field_name': field_name,
        'conversion': conversion,
        'format_spec': format_spec
}

printf_re = re.compile(
    '(?:' + replacement_field + '|'
        '%((?:(?P<ord>\d+)\$|\((?P<key>\w+)\))?(?P<fullvar>[+#-]*(?:\d+)?'
            '(?:\.\d+)?(hh\|h\|l\|ll)?(?P<type>[\w%])))'
    ')'
)

Well, with the above, I was able to parse almost all the cases discussed here except for this one:

import datetime
d = datetime.datetime(2010, 7, 4, 12, 15, 58)
s = '{:%Y-%m-%d %H:%M:%S}'.format(d)

I was not sure how I could fit the above case to my regex. After some discussions in #python on IRC, I found some limitations of regular expressions and that it is not Turing complete. People suggested me to use some parser tools.

I, being a strong supporter of "Never re invent the wheel", gave another shot to find some existing solution and lucky I was to come across _formatter_parser() of a Python string object.  It correctly found all replacement fields in python format strings properly and returned  an iterable of tuples (literal_textfield_nameformat_specconversion). All I needed then was to convert this info to a list of replacement fields in a format string. A simple script below would is all that I needed to extract replacement fields in a format string in Python:

replacement_fields = []

s = "{foo:^+30f} bar {0} foo {} {time:%Y-%m-%d %H:%M:%S}"

for literal_text, field_name, format_spec, conversion in \
        s._formatter_parser():
    if field_name is not None:
        replacement_field = field_name
        if conversion is not None:
            replacement_field += '!' + conversion
        if format_spec:
            replacement_field += ':' + format_spec
        replacement_field = '{' + replacement_field + '}'
        replacement_fields.append(replacement_field)
print replacement_fields
["{foo:^+30f}", "{0}", "{}", "{time:%Y-%m-%d %H:%M:%S}"]

That's all. Simple and easy, isn't it?

Comments



It's more than a year now that I have been working at Transifex. It's a great experience to be a part of the Transifex team. Well, it's been a roller coaster ride for me at Transifex. I had to go through steep learning curves, work with new stuffs, deliver great features, meet strict deadlines. It was fun, because of being part of an awesome team. I am very much thankful to Apostolis, Konstantinos, John and Diego for guiding me and helping me.

During the course of this journey, I have made many friends. It's always great to speak with the OpenTranslators folk. They all rock. I am also very much thankful to Youversion, Eventbrite, Pinterest, Dropbox folks for their queries and feedback. It makes me happy to help resolve issues with Transifex and add new features for our users. It gives me a sense of satisfaction that I am able to play my part for the greater good.

Mozilla has always fascinated me since a long time. I love Mozilla, it's logos, it's products and it's goals. At Transifex, I worked on developing new API features to support Pontoon use Transifex as a backend. Following this, I also got involved with Pontoon's development and have made a few commits. Finally, I contributed in some way to a Mozilla project.

A long road lies ahead. I hope it to be full of new challenges and excitement.

Comments



Yesterday, I was working on adding app specific loggers in Transifex. By app specific logger I mean a logger which shows the app name which generated the log. As of now, the logs in Transifex look something like this:

2012-06-29 13:01:43,300 tx DEBUG Saved: Project Avant Window Navigator
2012-06-29 13:01:43,312 tx DEBUG Saved: Project Switchdesk
2012-06-29 13:01:43,324 tx DEBUG Saved: Project Usermode
2012-06-29 13:01:43,342 tx DEBUG Saved: Project desktop-effects
2012-06-29 13:01:43,349 tx DEBUG Saved: Project im-chooser
2012-06-29 13:01:43,355 tx DEBUG Saved: Project Test Project
2012-06-29 13:01:43,364 tx DEBUG Saved: Project Test Private Project
2012-06-29 13:01:45,704 tx DEBUG Saved: Project Test Project
2012-06-29 13:01:45,717 tx DEBUG Saved: Project Test Private Project
2012-06-29 13:01:45,731 tx DEBUG Resource Resource1: New ResourcePriority created.

It does not tell anything about which app generated the logs. In a first glance, fixing this looks pretty straight forward and dumb. All it needs it to customize this https://github.com/transifex/transifex/tree/devel/transifex/txcommon/log module for each app and instead of importing the logger from txcommon.log, import it from the log module inside the app.

But this would lead to a lot of code duplication and a lot of boring changes in the code. So, I decided to customize transifex.txcommon.log module itself so that it can detect the function calling the logger. It was pretty straight forward to do this for the handler at https://github.com/transifex/transifex/blob/devel/transifex/txcommon/log/receivers.py#L6: def model_named() in the following way:

import re

tx_module_regex = re.compile(
    r'transifex(\.addons)?\.(?P<app_name>\w+)(\..*)?')
def model_named(sender, message='', **kwargs):
    """
    Receive signals for objects with a .name attribute.
    """
    from txcommon.log import _logger as logger
    sender_module = sender.__module__
    m = tx_module_regex.search(sender_module)
    app_name = '.' + m.group('app_name') if m else ''
    logger.name = 'tx' + app_name
    obj = kwargs['instance']
    logger.debug("%(msg)s %(obj)s %(name)s" %
            {'msg': message,
            'obj': sender.__name__,
            'name': getattr(obj, 'name', '')})

sender is the object or instance for which the log is being generated. In our case, it's a model instance. So, sender.module gives the parent module for sender. Using regular expressions, we extract the app name from the module name and we set the name of the logger as 'tx.<app_name>'. And we are done here (for now)! But when we do something like

from transifex.txcommon.log import logger
logger.debug('foo bar')

we do not have a sender instance to allow us to find the calling module name. After some searching, I found about the inspect python module. And all I needed was inspect.stack(). Here's what I did in https://github.com/transifex/transifex/tree/devel/transifex/txcommon/log/init.py:

  1. Write a wrapper around logger instance,
  2. find the caller calling the logger using stack.inspect(),
  3. accordingly set the logger name,
  4. and finally, log the event.
import logging, re, inspect
_logger = logging.getLogger('tx')

# regex to extract app name from a file path to a TXC app
tx_app_path_regex = re.compile(
    r'txc/transifex(/addons)?/(?P&lt;app_name&gt;\w+)/(\..*)?')


class Logger:
    """
    A wrapper class around _logger. This is used to log events
    along with app names.
    """

    @classmethod
    def get_app_name_from_path(cls, path):
        """
        Extracts app name from a file path to a TXC app

        Args:
            path: A string for the file path

        Returns:
            A string for the app name or ''
        """
        m = tx_app_path_regex.search(path)
        return m.group('app_name') if m else ''

    @classmethod
    def set_logger_name(cls):
        """
        Sets logger name to show calling app's name.
        """
        # inspect.stack()[2] since cls.debug() method has now become the
        # immediate caller in of this method in the stack. We want the caller
        # of cls.debug() or other logging method wrappers.
        caller_module_path = inspect.stack()[2][1]
        app_name = cls.get_app_name_from_path(caller_module_path)
        _logger.name = 'tx' + '.%s' % app_name if app_name else ''

    @classmethod
    def debug(cls, *args, **kwargs):
        """Wrapper for _logger.debug"""
        cls.set_logger_name()
        _logger.debug(*args, **kwargs)


    # And similarly for other logger methods like info(), waring(), error(), critical()


logger = Logger

Now, this is sweet! No one need to bother about logging events with app names. I am saved from editing hundreds of files and duplicating code ;) It's transparent and scalable. The logs now seem like:

2012-06-29 20:39:03,635 tx.projects DEBUG Saved: Project Foo Project
2012-06-29 20:39:05,575 tx.projects DEBUG Saved: Project Avant Window Navigator
2012-06-29 20:39:05,587 tx.projects DEBUG Saved: Project Switchdesk
2012-06-29 20:39:05,599 tx.projects DEBUG Saved: Project Usermode
2012-06-29 20:39:05,612 tx.projects DEBUG Saved: Project desktop-effects
..........
..........
..........
2012-06-29 22:15:07,088 tx.webhooks DEBUG Project project1 has no web hooks
2012-06-29 22:15:07,177 tx.releases DEBUG Deleted: ReleaseNotifications
2012-06-29 22:15:07,177 tx.releases DEBUG Deleted: Release All Resources
2012-06-29 22:15:07,466 tx.txcommon DEBUG Running low-level command 'msgfmt -o /dev/null --check-format --check-domain -'
2012-06-29 22:15:07,469 tx.txcommon DEBUG CWD: '/home/rtnpro/transifex/rtnpro/github/txc/transifex'
2012-06-29 22:15:07,661 tx.releases DEBUG release: Checking string freeze breakage.
2012-06-29 22:15:07,702 tx.resources DEBUG resource: Checking if resource translation is fully reviewed: Test Project: Resource1 (pt_BR)
2012-06-29 22:15:07,707 tx.webhooks DEBUG Project project1 has no web hooks
2012-06-29 22:15:07,740 tx.resources DEBUG resource: Checking if resource translation is fully reviewed: Test Project: Resource1 (ar)
2012-06-29 22:15:07,745 tx.webhooks DEBUG Project project1 has no web hooks

Thanks for reading. If you have any suggestions or query, please feel free to comment.

Comments



The 3rd day of FUDCON KL started a bit sluggishly for me. May be because of brainstorming and hacking till late night. We (Kushal, Soumya and me) decided to work on a new app to display system logs in a user friendly manner. We named the application Tower log tower, in short, tlogt, after Twin towers of Kuala Lumpur ;)

During the first few hours of the day, we went to visit some tourist spots in Kuala Lumpur: Aquaria and Petronas towers. After we returned, we settled down for the on going talks. Amidst of various talks on the 3rd day of FUDCON, I was sometimes in listening mode, but for most time I was in coding mode. We decided to try something different in TlogT. The UI would be rendered by a Django daemon with all the WOW factor of HTML, CSS and JS. I was to code the Django server code, while Kushal and Soumya were working on writing the parsers for extracting the logs for various processes. In a few hours, we had a decent Django based functional desktop app ready. Although, quite some work remains to be done on the UI part.

There was Kushal's session on Python for newbies in the afternoon. It's always nice to see Kushal talk on Python. I don't have the exact count, but I am sure that Python sessions by Kushal inspired many (including me and my friends) to start coding in Python. The ending keynote for the day was given by Abu Mansur Manaf. This should really boost newbies to become contributors :)

I spent the evening in the hotel room listening about various functional programming languages and features of the languages from hircus and Kushal. Later, we went out for dinner with the rest of the event crew members to a local food joint. I stayed up at night to see off other Kushal, Soumya and others who had to catch an early morning flight back to India. After bidding them good bye, I packed my bags and went to sleep.

Comments

Share