So, you’ve read my previous post and were bummed that the outlined procedure only worked for a Mac? I have good news for you. Here is an Adobe AIR application that can import your TweetDeck Groups into Twitter as lists:

TwitterListImporter.air [Update: See the note at the end of this post]

Download the application and install it. Because you already have TweetDeck (why else would you run this app‽‽‽) it’s not necessary to install the AIR runtime system. Once you start the application, you need to provide the path to the TweetDeck database file. The application will open the directory where you can then manually select the correct directory. In the directory that’s shown, locate a directory that starts with “TweetDeckFast” followed by a string of letters and numbers. Make sure you pick the directory, on the Mac there is also a plist file with the same name in that directory.

Once you’ve double-clicked on that directory, you will see a directory called “Local Store”, in which you will find the actual database file. It will start with “td_”, will contain your Twitter username and will end with “.db”. Select that file.

After specifying your Twitter username and password, just click on the “Convert” button and watch the show. After the application is done, you should have new Twitter lists with the members of your TweetDeck groups.

If you want to make any groups private, you need to do that on the Twitter web site.

Update – Note: When you download the file to a Windows system, the file may be named TwitterListImporter.zip instead of TwitterListImporter.air. Just rename the file. Windows is changing the extension – as far as I know, for no good reason.

Getting your Groups out of TweetDeck… Again!

If you are a regular on Twitter, you’ve probably heard about the new feature that is currently getting rolled out to more and more users: Lists. On the surface, they look pretty much like the TweetDeck groups, but once you start digging a bit deeper, there are a number of things that you can do with lists that are not possible with groups.

Even though the TweetDeck folks have indicated that lists will be added to TweetDeck soon, I did not want to wait, to get my TweetDeck groups into Twitter lists.

I’ve written a number of posts before that explained how to extract group information from TweetDeck and other Twitter applications, we can use these techniques to extract the group members and use the Twitter API to create new list and fill these lists with users.

This time, I will do this using the Mac version of TweetDeck – mainly because every Mac comes with all the parts that are necessary to accomplish this task (Perl, sqlite3), but also because I needed it for the Mac, and just did not have time to look at a Windows based solution yet.

Getting Our Hands Dirty

Download this Perl script, copy it to your Mac and rename it to TD2TL.pl:

• TD2TL

Then edit the file and provide your Twitter username and password – potentially you also need to modify the path to the TweetDeck database file. I’ve only tried this on a few TweetDeck installations, and it seems to work, but if things go wrong, find that file and modify that path as well.

The next step is easy: Just run the program from the command line by either making it executable, or by calling Perl with the program file on the command line:

chmod 755 ./TD2TL.pl
./TD2TL.pl

perl ./TD2TL.pl

While the script is running, it will report what it’s doing (e.g. creating a list, or adding users to a list). Once it’s done, you will have these groups available as Twitter lists – without the hassle to go through all your 1000 followed Tweeters and manually assign them to lists.

Update: If you are running the new Nambu 1.2 and are fed up with all the bugs, the SQL commands you need are slightly different. I’ll list them at the end. We now return to our regularly scheduled program…

A while ago I switched from TweetDeck to Nambu (via a few other stops in between). At fist I loved it, then I got used to it’s quirks, and now I’d like to move to something that actually works most of the time… Why is it so hard to write a good Twitter client?!?

Sorry, that rant wasn’t planned, it just happened… Let’s get back on track: As you may remember, a few days ago I posted instructions about how to extract group information from TweetDeck. Now that I’m considering ditching Nambu, I need a way to do the same with that application.

Background

After a little poking around the filesystem, I discovered that Nambu is also using the SQLite database system, even though it’s not an Adobe AIR application. The reason my be that it’s just using the database system comes with Mac OS X…

So, I dumped the database, and tried to find where the users and groups are stored in that database, and came up with a SQL command to extract that data the same way as with TweetDeck. Because Nambu is a Mac-only application, that makes these instructions Mac-only too. If you’ve already read my TweetDeck instructions, you will find a few things that I’m duplicating here for the benefit of new readers.

Extracting the Data

The process requires a little bit of post procesing (e.g. in a spread sheet application like Numbers, OpenOffice.org or Excel), but once you are done, you will have access to the group and user data.

On the Mac, the sqlite3 application actually comes with the operating system, and you can start it’s interface in a terminal window by typing the following command:

sqlite3

You get out of the application by using the .quit command (the dot is important).

The Nambu database file is stored in ~/Library/Application Support/Nambu and the file is called Nambu.db

Now start the Termainal application and change to the directory that contains the Nambu.db file. In that directory create a new text file named sql.txt with the following content (if you followed my TweetDeck instructions, you will notice that the structure is the same, just the table and field names are different):

We are almost there…

Now go back to the command tool and run the following command

cat sql.txt | sqlite3 Nambu.db

This will print all the users who are in groups – you probably want that in a file so that you can import it into Excel:

cat sql.txt | sqlite3 Nambu.db > group_data.txt

That’s it. Import it into a spread sheet program (make sure that you select “|” as field delimiter character) and do whatever you want to do with your group data.

Update: When you upgrade to Nambu 1.2, the database gets updated too, and some of the tables and fields now have new names. For Nambu 1.2, you need to use the following SQL sequence instead:

SELECT ZTWITTERGROUP.ZNAME, ZTWITTERUSER.ZNAME, ZTWITTERUSER.ZSCREENNAME
FROM ZTWITTERUSER
JOIN Z_24USERS ON ZTWITTERUSER.Z_PK = Z_24USERS.Z_26USERS
JOIN ZTWITTERGROUP ON ZTWITTERGROUP.Z_PK = Z_24USERS.Z_24GROUP
GROUP BY ZTWITTERUSER.ZSCREENNAME
ORDER BY ZTWITTERGROUP.ZNAME;

No, I don’t want to get groups removed from TweetDeck, I want to extract the group information so that I can use it otherwise. By extracting group information, I mean to create a list of groups and users belonging to those groups.

You may want to do that just so that you have a backup outside of TweetDeck, or to configure another Twitter client with the same groups, or just because you can

Background

Fortunately we already have a theory of how TweetDeck stores data: It’s an Adobe AIR application, and any AIR application can store data in a SQLite database. So the assumption is that all configuration data (users, groups, searches) is stored in such a SQLite DB. We can easily verify that by looking (I’m using a Mac for that, if you are on a Windows system, I’ll get to that soon) in the TweetDeck preferences directory – on a Mac that is “Library/Preferences/TweetDeckFast.[random letter and numbers]/Local Store”. In that directory we find a file named td_26_[username].db – Bingo! That’s our database file, and a quick check with the “file” command actually confirms that:

file td_26_user.db
td_26_user.db: SQLite database (Version 3)

The following command will dump the complete content of the database (in form of SQL statements) to the terminal:

echo .dump | sqlite3 ./td_26_user.db

Extracting the Data

Because most of my audience probably is not trying to do this on a Mac, I will now switch my instructions to Windows. If you need more information about how to do this on a Mac, please let me know in the comments.

The process requires a little bit of post procesing (e.g. in Excel), but once you are done, you will have access to the data.

On the Mac, the sqlite3 application actually comes with the operating system, for a Windows environment however, you need to download this file:

http://www.sqlite.org/sqlite-3_6_14_1.zip

Unzip the file and put the sqlite3.exe executable somewhere you can easily access it (e.g. I have a c:\temp directory, the path is short, easy to remember and easy to type. If you want to use this tool more often, you may want to store in in a directory where the operating system looks for executables.

Now you have to find out where the TweetDeck config files are stored. On my Windows system (I’m still running XP) it’s “C:\Documents and Settings\[user]\Application Data\TweetDeckFast.[random string]\Local Store”

You should find a file that starts with td_26 in that directory – that’s the TweetDeck database.

Now start a command tool and change to the directory that contains the td_26… file. In that directory create a new text file named sql.txt with the following content:

We are almost there…

Now go back to the command tool and run the following command

type sql.txt | c:\temp\sqlite3.exe td_26*.db

cat sql.txt | sqlite3 td26*.db

type sql.txt | c:\temp\sqlite3.exe td_26*.db > group_data.txt

Update (06/06/2009): Kathy Gill over at WiredPen.com has collected some information about where and how to get access to the TweetDeck database file on different systems. On Windows, you have to do a few more things to find the file:
How To: Back Up TweetDeck Groups and Preferences

I’ve been on Twitter for a while now, and over time I used a lot of different tools to make working with the twitterverse a bit easier. This post is not about the tools I’ve written, but the tools I use.

3rd Party Clients

Once you get past a handful of people you follow, taming the flood of tweets in your timeline becomes a fulltime job… or a 24/7 time waster. Very early on, I realized that I would need a Twitter client that allows me to create groups. This means that I divide the people I follow up into at least three groups:

• friends and family – the people I don’t want to miss a single tweet from
• the 2nd line of friends and people who have interesting things to say – but I wont panic if I miss a couple of tweets from them
• the “others” – these are the tweets I resort to when I’m really bored…

I first looked at Tweetdeck, and for a while that seemed like the right solution, but I’ve been running into more and more problems with it. It’s a memory and performance hog. Also, when I have to restart it, I lose important tweets.

Then Seesmic Desktop came along. It’s – just like Tweetdeck – an Adobe AIR application, but is bit more gentle with your system’s memory. However, I don’t like the way I need to configure groups. It’s one user at a time.

I really like Tweetie for the Mac – if it just would support groups, that would be my number one 3rd party Twitter client. No other client makes it as simple to see an ongoing conversation – just double-click on a tweet, and Tweetie will display it.

Right now I’m using Nambu. It’s not perfect, but until Tweetie comes with group support, I’ll probably stick with it.

When I’m using the Twitter web interface, I use Firefox with Greasemonkey installed – that allows me to use “Troys Twitter Script“. It does a lot of things that makes the web interface much more usable:

• show a conversation
• expand short URLs
• rudimentary group support
• embedded images and YouTube videos
• name completion
• …

Twitter On The Go

I use the original Tweetie – the one for the iPhone when I’m not at my “big” computer.

Web-based Twitter Tools

When you google “Twitter Tools”, you find list of the best 10 or 100 or 1000 tools… I’ve tried a few, and most of them have some entertainment potential (hey, who does not want to know their Twitter Grade?!? But how often to you need to really check if your grade went up or down a bit. Ask your followers about how good a tweeter you are.

I use a couple of tools that I think are valuable.

Topify makes it easy to follow or block new followers, and to send and receive DM’s via email. Go to their web site and watch the video, it’s good stuff. To make things easier, new-follower emails from Topify contain the MrTweet information about a user. This allows me to usually make a quick decision about following somebody back, or not.

Even though Topify helps me to follow back interesting users, I still have a bunch of older followers to take care of. Tweepler helps me to just go down a list of followers, and, based on data – like number of followers, following and how many tweets per day – I can then decide if I want to follow back.

If you want to export your followers or followees, use FriendOrFollow – it can export all the important information about users in a CSV file that can be imported into Excel, and then you can sort it by whatever criteria you find useful.

I’ve already mentioned MrTweet as part of the Topify follower emails. I also use it occasionally to get a better idea of who somebody is on Twitter. MrTweet can also recommend a user based on recommendations by others. If you like my tweets, recommend me on MrTweet…

So, that’s it. If you have any tools you want to share, please do so in the comments.