Let’s see i can explain this project very well or not. It started on Apr 14, 2015. When i wrote on my journal using Bahasa

digested - A developer-friendly newsletter

Lagi-lagi ada ide meledak-ledak di kepala gue. Tapi kali ini menarik sih jadi pantes gue catet. Oke, berawal dari ide yang “AI-powered curator”, secara singkat startup ini membuat sebuah program pintar yang mengautomasi kerjaan curator. Mengumpulkan berita, menyimpulkan lalu membagikannya.

Sebut saja nama startup ini Curatorium. Nah untuk membuat Curatorium aku harus bereksperimen dulu. Membuat sebuah weekly newsletter yang developer-friendly. Namanya “digests - A developer-friendly newsletter”. Tujuan eksperimen ini adalah untuk membuat curator berbasis kecerdasan buatan.

Secara kasar ada 3 fase:

1. Collect (lewat RSS dan lain-lain)
2. Curate (disini algoritmanya)
3. Publish (newsletter, tweet dan lain-lain)

Kalo ini eksperimen ini berhasil maka aku bisa dapat keuntungan:

1. Punya media (newsletter)
2. Punya startup masa depan

Secara singkat nanti digests akan dibagi dalam interest. Misal docker (digets.github.io/docker), interest docker akan punya akun twitter sendiri. Misal @dockerdigests. Kita harus buat juga automasi untuk mengtweet beritanya. Nanti link yang di share berupa URL digests.github.io/8678 redirect ke link asli. Mirip  quartz lah ya. Tampilan ala-ala monospace gitu.

Langkah awal:

1. Selesain kewajiban debelumnya
2. Buat collector RSS
3. Algoritma curate nya
4. Publish => via tweet, email, etc.

As you may see that i have a dream to build “AI-powered Curator” that could change the way the curator work. I want to build an Artificial Intelligence system that perform like curator did. Collect data from source[^1], using Machine Learning algorithm to classify raw data into categorial topic, then publish[^2] a specified data. It called “Curatorium”.

But in order to build Curatorium, I need to prove that the concept is work and to make sure the algorithm is ready for production use. So i started experimenting Curatorium system on project called “Digests”.

On that time, the mid-semester exam is going. So i just wrote what’s on my mind and not yet implemented.

Then, a few days later in Apr 20. I wrote on my journal

Codename digests changes to thedays.co (TD)

Status project TDnya masih dalam fase 1: Collect. Kita bahas dulu apasih tujuan collect disini?

1. Mengumpulkan data dari sources
2. Memastikan data selalu uptodate

Sekarang bagaimana cara mengambil data dari sources? Karena mengambil data dari source yang berbeda-beda kesulitannya disini adalah nentuin struktur dara program Go nya.

Kita mulai dari awal dulu deh

1. Gimana cara nambahin source e TD?
2. Bagaimana cara fetch data dari source?
3. Data yang di fetch mau buat apa?
4. Data yang diambil dari source ini berupa apa sih?

I change the codename “digests” to “The days” in order to build a good network’s name dockerdays, golangdays, swiftdays etc.

Let us assume that the sources type is only RSS/Atom feeds. I think the best way to create a data structure is based what we want to know from that entity. So we need to address the following questions to define the structure of our sources table

1. Where we can fetch the feeds?
2. Who add the source?
3. When the source added?
4. When the source updated?
5. When the source last fetched?

And for now, i think it’s enough.

Lately, I have been trying to build one of The Day component called “Collector”, which collects data from sources and make sure the sources are stay up-to-date.

I planned to use a bunch of Heroku Worker to build the collector. But, I can’t create more than 5 apps due to my account is not verified. In order to verify my Heroku account I create new bank account and use Veirified by Visa feature.

I tough Verified by Visa is can be used as payment method on Heroku, Amazon and verify Paypal account. After waiting about 3 days, it turns out that it can’t be used to neither payment method nor verify my account. ARGH!

While I’m developing  Syndication, I stumbled upon RSS board specification and found a  Superfeedr ads. I click on it and I found it is very useful for me. Now, Instead of create a Collector from scratch I choose to use Superfeedr which perform exactly like the collector which i have planed but more better. Superfeedr have a bunch of feature that i needed.

The best part is, it is Free! So i can’t wait to implement it!