Logdy

Appearance

From 0 to 500 stars: Logdy GitHub journey in 3 months ​

In this post, I'm going to describe how Logdy GitHub repository reached over 500 stars in 3 months. Starting from scratch with zero followers, subscribers and only a problem that seems to be interesting to solve.

This article can be a great insight if you're planning to start an open-source project and discover how to promote it. I hope I will at least inspire some people.

Build now!

I want to encourage everyone who thinks about starting a project on their own, whether it's a side hustle or OSS contribution - do it now!

The problem ​

Let me start with some motivation behind building Logdy. It all started with messy terminal logs during development, the one that is very verbose and detailed, but at the same time too noisy when it comes to finding anything.

I started to wonder, why there are so many tools that accommodate logs on production but not much when it comes to development time. In the end, a fast feedback loop is one of the key drivers of fast development time. Debugging and finding information can become a bottleneck when fixing bugs or solving a mystery case.

Of course, in the true spirit of TDD, I am spending the time on test development, there, I have a debugger and a very limited surface of the code that is being invoked at the moment. The problem arises whenever I start working on the frontend with a real backend setup or run some scenarios through the UI. The frontend app will send multiple requests which will end up with multiple lines of raw logs in my terminal during development time.

Over the course of the years, I tried multiple tools and colored terminal output. These all tools however proved to be incapable or inflexible of serving all of the use cases and at some point were dropped by myself. The only time a tool like this was useful was back in the days when I was working with Symfony framework in PHP and their profile/debug toolbar. This tool was really valuable, however, as the industry moved to SPA apps, the idea vanished. In addition that tool was tightly coupled with Symfony and there was no way to use it with other languages and tools.

The idea ​

So I started to wonder, why are we still using a terminal as a primary tool to work with text output when we have web browsers at hand, that can accommodate any kind of UI, these days mostly reactive and real-time. Combining that with low latency (because of the nature of the tool that would communicate with the web browser on a local host), should result in a smooth feedback loop.

That's how I got an idea for creating Logdy. I did a quick research and found a few alternatives, however, none of these offered what I needed (or what I imagined a tool like this could be doing).

I had a couple of main goals to build into Logdy:

  • Customizable Web UI - the tool needed to be able to accommodate any kind of logs and present them in a structured way (hackability)
  • Self-hosted & single binary - you just run it locally
  • Automatic faceted filters - for easier browse & search

The above in my opinion would suffice for the first version and would at least solve my problem.

Giving back to the OSS community

Building Logdy is also a great way to give back something to the open-source community. I have been professionally writing code for over 15 years and I was always using open source frameworks and libraries but I never gave anything back. I started to perceive this project as a way to contribute to the community and build something that can be useful to anyone.

2 weeks into creating the tool ​

So I started building the tool, the first version took me around 2 days of coding over the weekend and was looking very promising, I've spent a couple more days tweaking and adding more features to it and polishing the interface.

Sign up for updates about latest features in Logdy

It's a double opt-in, you'll receive a link to confirm subscription. We will only send you Logdy product updates

Next two paragraphs

You can skip to First steps into promotion if you're not into technical details of how the Logdy.dev homepage was built.

The homepage ​

This is where the story of Logdy starts and how I got into the whole idea of productizing the tool. As soon as I finished the first version of the tool I realized that building a README.md documentation in the GitHub repository may not be enough. I wanted the users to fully understand the capabilities of the tool. The choice was obvious, I needed a dedicated website where I could create any content I needed and build the documentation I wanted.

I knew at that time already that documentation was key. If the users are not aware of how to use the tool or what use cases it can serve, then the whole effort is for nothing. Truth be told, I knew that Logdy would have multiple applications and use cases so I wanted to highlight all of them.

As I was thinking about the homepage I quickly started considering that this should be a good source of traffic to get visitors. Previously I did not have any experience building open-source projects. So to me it was also a great fun to learn something new!

So I stepped onto preparing the GitHub repository with description, tags and a nice README.md. Not only that I had to decide how to build the documentation webpage. I checked a few open source tools for building the documentation. I ended up choosing something that looked simple but was also extensible - VitePress.

Why? It looked light, supported markdown and you could customize it with Vue (which is the frontend framework I know best). To this day this solution fills all of my needs and I was able to come across all the challenges. The best feature of VitePress is that I can embed custom Vue components directly within Markdown content. It's a great balance between readability and complexity.

GitHub Readme.md ​

Since I knew that logdy.dev website will be storing all of the documentation, I needed to think about a structure for Readme.md. I decided that Readme.md will emphasize the link to the docs and demo but will primarily focus on:

  • Installation steps
  • Quick start
  • Usage
  • Development / building for potential contributors

I left plenty of links to documentation and left it like that. I didn't want people to spend too much time but to switch to a website where I have greater control over the structure of the content and I can better explain what Logdy is. If you want to learn more about how to structure Readme.md I recommend this great article from Daytona

A few words about the infrastructure ​

Once I had the homepage and docs ready, I needed to choose how to host it. Since VitePress compiles to a static HTML/JS I knew a simple CDN is enough and would keep the costs low. After not much deliberation I chose Cloudflare Pages as I wanted to give it a try for some time now. The effect is great, it lets me host the whole Logdy.dev website for free with SSL certificates and global CDN.

With a little effort, I was able to set up a simple GH action that compiles and deploys an updated website to Cloudflare. I'm also using Cloudflare workers for handling mailing list subscriptions and gathering statistics for installations. Cloudflare workers also power the backend for new release notifications that you can see in the terminal once a new version of Logdy is available.

Logdy demo ​

Since Logdy has many features and multiple applications, I knew it could be hard to imagine how it works without installing it. But since its architecture is fairly simple, I was able to deploy a demo.logdy.dev and let potential users play with it. I feel this only positively impact potential interest in the tool, on one hand, it can spark interest on the other hand, if somebody is totally not interested, they don't have to even bother downloading it.

First steps into promotion ​

My idea was to start slow at first, get a few users and their feedback and work my way through to polish Logdy. I posted a few posts on Reddit, Discord channels, and tried to add Logdy to a couple of awesome lists on GitHub.

GitHub a word about awesome lists

One observation I have is that a lot of "awesome" lists on GitHub are no longer maintained. I saw a couple of lists where there were PRs from the past few months and nobody was accepting any new contributions.

The above did not bring the effect I was expecting. Clearly, not having any kind of audience is not helping to get a few initial users.

Hacker News post ​

After a few days, I decided that I was going to post to HN since I didn't have any more ideas on how to promote the project. It was spot on.

Logdy got a very good reception and within 2 days the project got its first 170 stars on GitHub.

Next steps ​

As the project started taking off, it kept me motivated to add new features and polish existing ones. During that time I also have been slowly building the website with documentation and posts that would eventually get indexed by Google.

I visit Hacker News daily, after the launch post I kept being active in the "Show HN" category and have been actively commenting on the threads added there. As time passed I realized I could recommend Logdy whenever it's possible. That's how I wrote a few comments that ended up with some upvotes.

I wasn't sure in the first place if this is something I should keep doing or not, the effects were nice. I decided to only promote Logdy through justified cause, like when somebody asks for a tool like this or the topic is connected to the same area.

Blog and technical articles ​

Another thing is that I write blog posts and extensive documentation which you can see on this website. This slowly builds visibility in the search engines and drives traffic there.

As you can see, I focused on the potential applications where Logdy can be useful, I mix it with languages and technologies as usually people will search for a bit more specific things. Let's take PM2 as an example (great process manager written in Node.js), there is a command pm2 logs which I bet people often use to browse their logs, however, it's CLI and maybe you would prefer something with a UI but you don't have any third party logging service like DataDog or similar. This is where Logdy fits perfectly. Another use case is for Docker, whether running locally or remotely, Logdy can provide a nice UI for logs.

Clearly there is plenty of space to write good quality SEO articles and weave Logdy there.

I also posted a few articles on Medium and Dev.to, however, writing articles is a time-consuming task and I simply cannot afford too much time to focus on these services. In addition, I'm not entirely convinced that I should grow my audience on a 3rd party space. This should not be a core strategy as I don't want to be dependent on a single entity that decides whether my articles receive traffic or not. However, this direction is something I'd like to explore in the future. Right now, I'm only posting reprints, which are copies of the original domain but with a canonical link set to the original URL. This is important for Google so they know it's not duplicated content.

Homebrew and install script ​

This is not a big milestone but I believe that making it easy to use the product is key. That's why I went with Golang and a single binary in the first place. However, over time I realized that downloading a binary and then adding it to your PATH can still be cumbersome. So I decided to take a closer look at homebrew (adding a formula there is simple and automated!).

This is how you can install Logdy these days (still you can compile it yourself or download a precompiled binary).

bash
$ curl https://logdy.dev/install.sh | sh
# OR
$ brew install logdy

Letting users know about new releases ​

Immediately after the first release, I started considering how to keep adding features to Logdy but also update users about new releases so they don't fall back behind. This is not problematic if you host the app yourself, but if the users download and run it locally, it can be a source of headache. I simply solved this problem. Every time there is a new version available and the users don't yet use it, a message in CLI is displayed.

bash
$ cat file.log | logdy
Listen to stdin (from pipe)
WebUI started, visit http://127.0.0.1:8080    port=8080
----------------------------------------------------------
>                NEW LOGDY VERSION AVAILABLE
> Version: 0.10.0
> Date published: 04 April 2024
> Download: https://github.com/logdyhq/logdy-core/releases/tag/v0.10.0
> Blog: https://logdy.dev/blog/post/logdy-new-version-announcement-v010
> What's new? Traces and Correlation IDs
----------------------------------------------------------

Underneath, every time you invoke logdy command, it sends a GET request to https://update.logdy.dev?version=xxx to check whether a newer version is available. Simple and effective, the side effect is that I know exactly how many times the tool is being stared throughout the day.

Golang weekly feature (Updated May 11 2024) ​

Logdy was featured in Golang weekly issue #506. I chased Peter from Cooperpress for a couple of weeks, but eventually, he read my message and responded that Logdy would be featured in one of the next issues:

I like it! Should be in the next issue 😃

Couple of days later, the next Golang weekly lands in my mailbox, and there it is:

That was a great traffic boost, I got around 2,000 visitors on a website, which in turn transitioned into around 120 stars spread over a couple of days, since people open and read their emails even a couple of days after they receive it. It was also a nice boost for potential a links from other websites and sources as people reading about Logdy could link to it from their profiles/websites/blogs/etc.

I couldn't omit SEO in this article. Although I'm not knowledgeable of the SEO techniques, I don't know much about any sophisticated optimization techniques, I know that good quality content can promote the website, and that's enough for me. My main goal and priority for SEO is to build relevant content that can attract users searching for help on the web.

Here are a few statistics about the SEO quality of Logdy.dev domain so far.

Domain authority

MOZ:

Domain Authority (DA) is a search engine ranking score developed by Moz that predicts how likely a website is to rank in search engine result pages (SERPs). Domain Authority scores range from one to 100, with higher scores corresponding to greater likelihood of ranking.

These come from Ahrefs and Semrush free accounts. I was curious about the number of backlinks as the number is quite high and unexpected. Turns out, and this is not a surprise, there are a lot of automated HackerNews clones that list everything that gets published there. So these backlinks falls mainly into 3 categories:

  • HN clones/reprints (~40%)
  • Github clones/reprints/automated links to a project (~40%)
  • Actual mentions and link coming from humans (~20%, mostly telegram groups)

All in all, as to my limited SEO experience and knowledge, these numbers are not big traffic drivers, Logdy.dev domain is missing good backlinks from high authority domains. I don't mind lower domain authority backlinks, but I also hope that it will not harm the score of Logdy.dev domain.

Numbers & statistics ​

It's time for a small summary, all of the numbers have been collected since the first commit on January 26, 2024. The launch on Hacker News took place on February 6, 2024.

  • 575 stars on GitHub as of writing this (around 500 stars in 90 days since starting the project)

  • 130 email update subscribers

  • over 10k page visits

  • 430 clicks from SEO

  • 25 blog posts

  • over 1500 installs/downloads (GH stats + install.sh script + homebrew)

  • average 50 uses per day

  • 14 issues solved

  • 11 releases

Update 11 May 2024

Last few days were crazy after Golang weekly issue, all of the numbers are between 10-20% higher now.

TL;DR summary ​

To summarize, here's a short breakdown of the most crucial things you should focus on when promoting an open-source project.

  • Build a clean and detailed website
  • Write thorough documentation with code examples
  • Write a readable GitHub Readme.md
  • Invest time into writing technical articles
  • Show users how the tool works through a demo: images, screencast, actual product
  • Promote the tool within communities

I am very happy with the results so far. Overall, in 3 months' time I was able to build some audience and user base for Logdy, along with that, SEO efforts started to pick up and Logdy.dev is getting more traffic from search results each week. Patience will be key here, but since I'm not in a rush, I can slowly wait for the traction to build up even more.

Next stop: 1000 stars ​

So far, the journey has been great, I've learned a lot about building OSS project but also promoting and getting traction. I know GitHub stars are not a very reliable measure of any kind of metric besides "star", in the end, the usage is what matters and having a product used on average 50 times per day is not anything big, but it's also not nothing. I believe that if I maintain it, the usage will steadily grow. My next stop will be at 1000 stars. I'm not sure if I'll be changing anything significantly when it comes to my approach towards promoting Logdy. I'll keep focusing on posting useful SEO content, and animating Hacker News community and that's pretty much it. I'm also planning to start posting on X.com (formerly known as Twitter), however, I'm not convinced I can find potential users there.

What's ahead? ​

My next focus towards which I'd like to shift Logdy (while still keeping the current direction as well) is to make it easier to work with big files as it's currently not a well-covered area. Logdy was created with small, volatile data in mind, one that doesn't need to be stored and one that doesn't grow much so performance doesn't have to be a concern. However, as I'm getting more feedback and looking around, it seems that this application could be useful.

Another aspect I'd like to explore is the ability to embed Logdy or its UI into another web app or a microservice.

Third, I'm looking for ways to introduce collaboration and team aspect into the app, however still need to explore that space.

That's it, thanks for reading the entire blog post. If you would like to stay up to date, let me invite you to sign up for a mailing list below.

Best, Peter from Logdy (contact: peter (at) logdy.dev)

If you would like to hear from me when the next milestone is met (1k stars) subscribe to the mailing list below

Sign up for updates about latest features in Logdy

It's a double opt-in, you'll receive a link to confirm subscription. We will only send you Logdy product updates

Copyright © 2024-present Logdy