This is Volume 1 of the programming news feed. You can find Volume 2 here.

To know more about this feed, what & why, visit here. If you like the feed don’t forget to share it with your folks.

To stay notified on the updated content you can follow 8bitmen on Twitter & Facebook

 

Hexagonal Architecture – Using Ports & Adapters to Make the Software Loosely Coupled & Easier to Test  

A couple of days back I came across this term the Hexagonal Architecture. Had no clue what it was, did some reading. Eventually, sharing it with you folks.

The hexagonal architecture consists of three components Ports, Adapters & the Domain.

Hexagonal Software Architecture

The focus of this architecture is to make different components of the application, independent, loosely coupled, easy to test.

The application should be designed in a way such that it can be tested by humans, automated tests, with mock databases, mock middleware, with & without a UI, without making any changes to the code.

The architectural pattern holds the domain at its core, that’s the business logic. On the outside, the outer layer has Ports & Adapters.

Ports act like an API, as an interface. All the input to the app goes through the interface. So, the external entities don’t have any direct interaction with the domain, the business logic.

The adapter is the implementations of the interface. They convert the data obtained from the ports, to be processed by the business logic. The business logic lies at the centre isolated & all the input and output is at the edges of the structure.

The Hexagonal shape of the structure doesn’t has anything to do with the pattern, it’s just a visual representation of the architecture.

Initially, the architecture was called as the Ports and the Adapter pattern, later the name Hexagonal stuck. The Ports & the Adapter analogy comes from the computer ports, as they act as the input interface to the external devices & the Adapter converts the signals obtained from the ports to be processed by the chips inside.

Should you wish to read more about the hexagonal architecture this is a good resource.

Coming down to the real-world code implementation, isn’t that’s what we already do with the Layered Architecture approach? We have different layers in our applications, we have the Controller, then the Service Layer interface, Class implementations of the interface, the business logic goes in the Domain model, a bit in the Service, Business and the Repository classes.

Layers of a web application - 8bitmen.com

Well, yeah. That’s right. First up, I would say that the Hexagonal approach is an evolution of the layered architecture. It’s not entirely different.

As long as the business logic stays in one place, things should be fine. The issue with the layered approach is, often large repos end up with too many layers besides the regular service, repo & business ones.

The business logic gets scattered across the layers making testing, refactoring & pluggability of new entities difficult. Remember the stored procedures in the databases & the business logic coupled with the UI with JSP Java Server Pages?

When using JSPs and stored procedures, we still have the layered architecture, the UI layer is separate, the persistence layer is separate but the business logic is tightly coupled with the layers.

On the contrary, the Hexagonal pattern has its stance pretty clear, there is an inside component which holds the business logic & then the outside layer, the Ports & the Adapters which involve the databases, message queues, APIs & stuff.

This is a good resource on layered vs hexagonal pattern.  Here is a GitHub resource on understanding the architecture with the help of a project.

Here is another project on GitHub on hexagonal architecture.

 

Mob Programming

Mob programming is a programming approach where the entire team works on the same thing, at the same time at the same place, on the same computer.

Well, this is interesting, it’s like a head-on approach towards speeding up with the execution of super boring tasks assigned to a poor solo developer which he doesn’t want to do.

I can imagine everyone standing and staring at the boring tedious task with lasers coming out of their eyes. And the molecules of the super boring task start to disintegrate until the task completely disappears. Mob programming sounds fun.

 

Remote Teams Are On – Distributed Teams Help Startups Move Faster

If you have noticed lately, the number of job openings for remote programmers have a continual rise.
Primarily because it’s a win-win scenario for everyone. Developers don’t have to get stuck in traffic, they can work with the comfort of the homes. They are comparatively more productive.

The distributed communication tools are doing an amazing job of connecting people across the world, living in different time-zones.

Here is what VentureBeat has to say about it. A Billion $ valuation software company with 800 employees operates without any office.

StackOverflow believes in remote work & is pretty successful with it.

 

Airbnb & Udacity Abandon React-Native for Native Tech – Native Vs Hybrid Apps

Airbnb engineering in a series of blog posts share their experience of writing mobile apps with React-Native.

React-native lets us write mobile apps which appear like native apps using JavaScript.  Just to give you a little background on this, native apps are apps built using the native language of the platform, for instance, Objective-C for IOS, Java, Kotlin for Android.

Native apps are preferred by businesses simply because of their performance. Also, they have full access to the device hardware, native APIs, the interface is fluid, the animations render pretty smooth.

A few downsides when writing native apps is, we have to maintain separate codebases for every platform. A separate codebase for IOS, Android, Windows etc. Different platforms different apps. This process is expensive for obvious reasons. Also, the native tech has a learning curve.

Developers get attracted to frameworks like React-native, Cordova as they enable us to write cross-platform apps using only JavaScript.

The apps written with React-Native, also known as hybrid apps, appear as native apps, engineering teams don’t have to maintain separate codebases for different platforms. Developers working on the web are acquainted with atleast basic JavaScript in a majority of the cases.

Hybrid apps are not quite as smooth as native apps in terms of performance & rendering & having access to the native APIs of the hardware. If you are writing a game, you are going to have a tough time achieving the desired performance with a hybrid app.

So, the trade-off conundrum between spending and application performance persists.

Airbnb

Since the Airbnb engineering team built their website using React, they considered React-Native as an opportunity to speed up the app development process by having one solo repo as opposed to having dedicated platform codebases.

They spent a couple of years working on it & eventually abandoned React-Native for the native technology. They shared insights into the pros & cons of writing a hybrid mobile app.

Ditching React-native does not mean it’s not powerful enough for writing mobile apps. It’s just that it did not fit a few of the required use cases.

If the app requirements are simple, React-native is a pretty powerful tech which could save us a lot of time & money.

The dev team at Airbnb was able to write features for cross-platform with upto 95 to 100% of the shared code & 0.2% of the platform-specific code. They used Redux for state management.

The animations such as scrolling & parallax were written using the React-Native animation library.  Since the code was in JavaScript they were able to leverage other open source JavaScript project such as Redux, Jest, Re etc.

The primary issues when working with React-Native were that the framework isn’t as mature as the Native tech Android, IOS. They had to write a lot of patches to the React-Native library to get the desired functionality.

They found some of the trivial stuff when could be easily done with native tech, quite difficult to pull off with React-Native.

The lack of type safety in JavaScript made it difficult to scale & turned out to be difficult for engineers who were used to write code in languages with default type-safety checks. The lack of type safety made code refactoring extremely difficult. For a full account of the issues with React-native here you go.

Udacity

Here is another instance, where the Udacity engineering team abandoned React-Native due to the increased number of Android-specific features requests by the users.

Their android team was reluctant to go ahead with the hybrid app approach, the long-term maintenance costs of the React-Native codebase were high. They also faced UX consistency issues across the platforms.

Facebook

In the past, Facebook has too invested quite some efforts into writing their mobile app with Html5 instead of the native tech & later admitted that it was a mistake.

Hybrid Vs Native What to Pick?

I am more of a backend developer who has worked on Full-Stack, written front-end code using JavaScript, React. I also have written a couple of mobile apps on the Android platform.

So, I think I am qualified to give an opinion on this. First thing, I would say is this is a trade-off. Also, it depends on the use case to some extent.

We just can’t discard hybrid tech on the grounds of performance and the availability of the other native features. If we have simple application requirements, for instance, a news app, hybrid tech is gonna save us a lot of time and money.

For use cases such as gaming apps, finance, fancy social media apps with animations & stuff, apps which have a lot of features & demand some serious computational power from the hardware. I would always pick the native tech stack.  

 

Replicating Bugs in Distributed Systems, Microservices – Distributed Tracing

It’s kind of manageable to replicate bugs and fix them when working with solo monolithic codebases. But what if a feature is powered by a distributed system with several different microservices running on the backend?

In this scenario, looking into one microservice does not give the complete picture. Fixing stuff across microservices is one of the hardest things to pull off.

Distributed tracing is a mechanism which helps us trace and fix stuff in complex microservice environments. It helps us track requests, timing & the metadata of the data flow throughout the system.

8bitmen.com Microservices architecture

I once had to fix an intermittent authentication issue across two microservices. Data from one flowed to the other, the second microservices often failed to authenticate the information incoming from the first microservice.

I spent days just trying to get to the bottom of the issue. It was hard, going through logs and putting debugs everywhere.

Finally tracked the issue, it was with the threads, sometimes the auth info got copied on to a local thread, sometimes it didn’t. The code was in Java, it used the Executor framework. I had to tweak the thread code to fix the issue. A distributed tracing framework might have come handy in that situation.

If you are intrigued further, you can look into Zipkin, it’s an open source distributed tracing system.

 

Looking For a Fast Front-end Programming Language? Checkout Elm

Elm is a functional programming language which compiles to an optimized version of JavaScript. It’s something like React JS for building web apps & websites.

Why even think about Elm when we already have React, Angular?

Elm is fast. It focuses on simplicity & ease of use. There are no null, object undefined, NaN, Uncaught TypeError errors rather the errors are readable, developer friendly.

Elm enforces a Model View Update architecture. It’s a well-architected code, stays well-architected even when the app grows.

Redux talks about Elm. Views of a Swedish startup on Elm. Hackernoon on Elm.

Uber’s visual insights team delivers a talk on Elm

 

What On Earth is a Kata Container?

We are all aware of the containers. Now, what is a Kata Container?

Well, it’s a secure container runtime with lightweight virtual machines which are just like containers but more secure. They leverage the hardware virtualization tech to isolate workloads.

The present containers have software based isolation they share the hardware. Kata containers provide the isolation on the hardware level too making things more secure. Here you go if you want to read more about it.

The name intrigued me pretty much when I heard it for the first time, thought will share it with you guys.

 

Developer Friendly Proxy Cloud Platforms

Cloud has become the norm for application deployment, it’s the preferred choice for everyone be it an indie developer, a startup or an enterprise. And this is for obvious reasons. Read Why Use Cloud? How is it Different from Traditional Computing?

But often we tend to get overwhelmed by the sheer number of services & options offered by the cloud giants in the industry. It can be intimidating sometimes for a developer with not so much DevOps experience who just simply want to deploy his code & be happy.

We as developers just want to focus on deploying our code & hitting the url in the browser instead of investing time in doing the research about the right service, infrastructure etc.

This led to the emergence of a new type of cloud platforms I would call them the developer-friendly proxy cloud platforms which further simplify the development process. They are more developer friendly, easy to use, simple. Ease of use is the USP of these platforms.

One good example of this is Heroku. It’s a PaaS Platform as a Service product built on AWS. It enables the developers to not worry about infrastructure, application servers, runtime environment etc.

Here is good discussion on StackOverflow on deploying apps on Heroku vs AWS.

A few more similar platforms I’ve come across are Netlify, Zeit.

Besides the ease of use, these proxy cloud platforms are more dedicated towards specific use cases. Their pricing plans are more customized toward those use cases.

Netlify focuses on providing hosting & serverless backend for static websites.

Zeit is a dedicated serverless cloud platform as a service.

A good use case where I think these dedicated platforms seem fitting is when we are starting up either solo or in a small team & need to focus on building features, getting the MVP out, keep evolving it in fast iterations, & not worry about the infrastructure, investing time in research.

In the starting phases, a business can hardly afford a DevOps team.

 

Interested in Saving Birds?

By any chance, if you are interested in saving birds here is a really cool open source project called The Cacophony Project. 

The project keeps a track on the bird predators like rats, possums etc via machine learning algorithms, thermal imaging, monitor the bird’s songs to measure the project impact in an area.

It is built using embedded systems, a web-based backend, machine learning pipeline. The technologies include Raspberry Pi 3, Node JS, PostgreSQL, Minio – an object store for AI,  the test code is written in Python.

Check out the project on GitHub.

 

Building a Scalable Search Architecture

For basic search use cases, our old school SQL databases suffice but for some additional search feature requirements, we have to look up to technologies dedicated & built for running scalable search queries such as Elasticsearch & Solr.

Elasticsearch has become the defacto search framework of the industry considering the number of features it offers out of the box.

When using Elasticsearch a typical design is to stream in the search-related data to the Elasticsearch document-oriented database along with persisting the data in the database.

But there is a catch in this, what if during the persistence transaction involving two different distributed databases the write fails in one? How would you handle the distributed transaction failure?

Here is a nice write-up by confluent designing a scalable search architecture with Apache Kafka & distributed data stores. Kafka acts as the middleware data pipeline between the systems facilitating smooth data flow by decoupling the systems and enabling them to run independently.

 

REST APIs Vs Streaming APIs

REST APIs are fundamental to the large scale system design and web architecture today. Streaming APIs enable the continuous streaming of data to the client over HTTP. Typical examples of this is a video streaming service, streaming text in a microblogging site like Twitter, an internet radio service, online massive multiplayer browser-based games etc.

How different are the streaming APIs to the REST APIs? Is the difference subtle or are they architecturally different? Let’s find out.

To implement a data stream between the client and the server we have to establish a persistent connection between them. This is made possible by technologies such as a Message Queue, an in-memory cache on the backend, & on the client with Web Sockets, Comet, Server-sent events, Long Polling, Html5 Event Source API, Stomp etc.

Rest API Vs Streamingn API Persistent connection

A REST API is stateless, it’s more like a conversation between the client & the server. Every time the client sends all the information needed to get the response from the server. APIs don’t save any state.

On the contrary, the Streaming APIs having a continual open connection with the client. Instead of the client polling for data every time, the request is sent for the first time by the client & the server pushes the data to the client whenever there are new updates available.

The connection between the client and the server stays open with the help of heartbeat interceptors. These are just blank requests response between the client and the server to prevent the browser from killing the connection.

Twitter uses a streaming API to push the data to the client.

 

Cold Start Problem With Serverless – a Trade-Off

Serverless has gained quite some popularity over time. The FaaS Functions as a Service offering is saving businesses quite some money, especially if your application is stateless. For instance a tax calculation function or an image optimizing function which has nothing to do with the state of the previous request. Just feed the input and get the response.

Serverless functions save money solely due to the reason that businesses are charged only for the computation, not for the allocated instances. As there are no instances reserved to businesses. The micro-containers are assigned dynamically. The provisioning is automatic.

This brings along an issue with it, the issue of Cold Start.

If you are not aware. Cold Start simply means the additional time the cloud platform takes in warming up instances for the request. This happens if the function hasn’t been hit for a while.

Though the warm-up time isn’t too much still the platform has to load the code from the persistent disk, spin up a container, load the code in memory and execute the code.

This might be ok for some businesses that can afford this delay but also not ok for others running mission-critical tasks.

A workaround is to either make an external monitoring service responsible for not letting the instances sleep or writing a CRON job in the code which runs periodically at stipulated intervals to keep the instances warm. 

The time after which idle instances go cold depends on the default configuration of the cloud provider also on the configurations set by the developer if any.

 

Cost Reduction & Open Security Strategies When Deploying Our Apps On the Cloud

Infoworld shares some insights on researching on third-party cloud security solutions besides the cloud-native security offered by the cloud platform for a more holistic & open security approach especially during multi-cloud deployments. Also, read about the upsides of buying reserved instances upfront in the cloud.

Cloud platforms today have marketplaces which offer ready to go solutions like technology stacks, services etc. which save businesses a lot of time. Averts the need for writing stuff from scratch or relying on the cloud platform for additional features.

Have a look at Google Cloud & AWS marketplace for more info.

 

Gitlab Ends Support for MySQL

Gitlab finally ended the support for MySQL due to several technical reasons. They used PostgreSQL & MySQL as their primary database. Majority of their customers were on PostgreSQL & a very small no was on MySQL.

The engineering team was facing problems making both the databases work together in conjunction. For instance, they wanted to use PostgreSQL’s Lateral Joins for optimizing the dashboard events but they couldn’t since this feature wasn’t available in MySQL.

Also, there was a lot of MySQL specific code which they had to manage which slowed them down.

 

[Industry] 75 million old jobs will go away by 2022, and 135 million new jobs will be created because of new technologies

Says Infosys president in a conversation with [email protected]. Tech has taken over there is no denying that it’s evolving at an unprecedented pace & the demand for software programmers in the industry is like never before.

And in order to not become a dinosaur, we have to be lifelong learners. Also, machines will take over the repetitive work & we will be more involved in doing the non-repetitive work. Which is pretty good if you ask me 🙂

 

PANE A Functional Programming Language with Visible Data

PANE is a live, functional programming environment built around data-visibility.

In PANE, all the intermediate values are visible by default, and we construct a program by acting on these concrete values. The language is built for general data processing.

Here is my understanding, In a PANE program at any stage the data would visible to the programmers by default just like we access it via the debug mode in other languages like Java or so. This is also known as the Principle of Radical Visibility.

The upsides of data visibility are, One – we can write faster efficient code since we are kinda aware right at the time when writing code that how the data flows through. And we can tweak our programs accordingly.

Second, let’s say we have an array of arrays or some input & we are not sure about what function to write to get the desired output. We provide PANE with the input and the desired output. PANE can figure out an apt function for us. This style is known as programming by example. You can run the demo here

 

Go Rises in Popularity – Especially When Implementing Concurrent Use Cases

Go programming language has a continual rise in the programming language popularity rankings of late. Primarily due to being a fitting tech when writing distributed systems.

Concurrency is a key thing when building distributed systems & Go really shines in implementing concurrent scenarios having an in-built concurrency mechanism.

I’ll just give a little background on Go. The language was built by Google to address some of the software infrastructural problems they were facing. They were having a hard time dealing with multicore processors, large scale computation clusters, infrastructural networking.

The code build time continually soared even on massive computation clusters.

Go was written to deal with all these scenarios and to make the development and production environment more productive.

It has an in-built concurrency, garbage collection & features to handle memory leaks which are critical to today’s software development landscape.

According to Google, Go is much better than C++ & Java in handling concurrency scenarios.

Also, the developers of Go were pretty motivated by this when writing the programming language.

Here is a good discussion on Go vs C++ on Reddit

Here is another Reddit discussion on using Go for writing distributed systems.

Some of the popular products written using Go are Kubernetes, Google uses Go on the Youtube backend & other services, Cockroach DB a strongly consistent horizontally scalable SQL database is written using Go, Bitcoin Lightning Network, Ethereum, Docker, Mattermost an open source alternative of Slack etc.

Having said this, I also want to state that there is no silver bullet, no perfect language, do your research having your use case in mind before picking up any language when building your product.

 

Concurrency Vs Parallelism

Now that I’ve been talking about Go, a programming language built for concurrent use cases. This is the perfect time to talk about Concurrency vs Parallelism.

What do you think is the difference between the two terms? Or there isn’t any? Or is it subtle?

When I first came across the two terms, I thought about it a bit. Hmmm…. Well, nothing came to my mind. I then re-thought about it in the context of web applications.

Concurrency is handling multiple users at one point in time by a web server whereas parallelism is executing the commands made by those concurrent users. Think of Ajax, running parallel user requests. Whereas concurrency is spawning separate threads by the server for every concurrent request.

The difference is subtle but it is there. Here is a good talk on Concurrency is not parallelism by Rob Pike

If you want to delve more into concurrent & parallel programming languages here is a list of them on Wikipedia

 

Facebook Engineering Team Open Sources StateService: A State Machine as a Service for Dev Ops Teams

All of the services at Facebook are designed to be resilient. They are built intelligent enough to recover automatically from network outages.

In case they lose connection with other services running concurrently in real time. The entire service as a whole doesn’t get affected by a few services, or a data centre going down.

All of the production services at Facebook are built in-house. They also integrate with several other third-party services which run in VMs. Bouncing back from a third-party service outage isn’t such a simple task & requires time to get things back to normal.

To tackle this Facebook wrote a solution StateService which helps reduce the disaster recovery time. Facebook eventually open sourced the service to the community that would help the solution grow even better. Here is the GitHub repo.

 

Assess the Microservices Architecture of Your Business, Get a Best Practices Insight into the Individual Services Running

Chris Richardson, the creator of Cloudfoundry.com published a Microservices architecture assessment based on his years of experience running online systems, also training & consulting various organizations around the globe, helping them scale.

The assessment helps us compare against & improve the architecture of the microservices running in our production environments. The assessment helps us contemplate on our application architecture, organization & process. It evaluates the current architecture, figures out weaknesses & help fix it.

 

An Insight into the CERN Labs Open Data Portal Backend Technology Stack

In a SuperUser article,  the technology lead behind the CERN Open Data portal talks about the backend technology stack of the portal. The portal holds petabytes of research data on particle physics & related stuff.

The portal runs on about 8 virtual machines running on CERN OpenStack cloud infrastructure. The machines are managed by Puppet & the portal runs on top of the Invenio repository framework.

The Invenio framework is an open source framework for managing large-scale digital repositories.

Redis is used as a caching solution & the database is SQL based.

 

[Industry Trends] How Integration Of Blockchain With Gaming is Helping Gamers, Developers & Publishers 

Implementation of blockchain & use of crypto tokens in games is gaining ground & this is not a bubble or a hype but for obvious reasons. Enabled by blockchain, gamers can carry around stuff, that is, items bought in one game to other games.

This averts the value of the items going null when or if a particular game goes out of business. Players can easily use the items bought in that game in other games.

You might have heard the term Multiverse in context to this. Multiverse simply means using items, bought in one gaming universe, in different gaming universes.

This also helps the businesses as users are not wary of the game being put offline by the game publisher & all their investments going void. They are more inclined to make the in-game microtransactions having the safety net of the blockchain.

 

[Career] Most In-Demand Skills Of 2019 by LinkedIn

Linked-In lists out the most in-demand skills of 2019. With Cloud computing, AI, UX design, Game Development, Animation, NLP & many more among them.

 

Code Your Way to A Smart Home with Raspberry Pi & Mozilla’s Things Getaway

This is one of the coolest things I’ve come across on the web. With Mozilla’s Things Getaway & Raspberry Pi we can control the devices on our home directly via the web from our phones or laptops.

Project Things by Mozilla is a framework of software and services that can bridge the communication gap between connected devices by giving “things” URLs on the web. Instead of having separate apps for controlling different devices. All the devices can be controlled from a single app.

Project Things focuses on three components:

The Things Gateway: This is an open source implementation that acts as a gateway. It bridges the IoT devices to the web.

Things Cloud – This is a collection of cloud services hosted by Mozilla. It helps with the management of quite a big number of IoT devices spread of a wide geographic area.

Things Framework – Things framework helps developers write modular, reusable components which further helps in creating IoT devices connecting directly to the Web.

In case you want to contribute to the project. The code is available on GitHub.

 

More On the Blog 

Facebook Real-time Chat Architecture Scaling With Over Multi-Billion Messages Daily

Instagram Architecture – How Does It Store & Search Billions of Images

How Does PayPal Processes Billions of Messages Per Day with Reactive Streams?

A Simple Guide to High Availability, HA Cluster Architecture & Fault Tolerance

Why Bitcoin Cannot Be Used as a Real Currency in Our Everyday Lives – Simply Explained