Latest Software Technology, Programming News, New Programming Languages Feed Vol.1
This is Volume 1 of the programming news feed. You can find Volume 2 here.
To know more about this feed, what & why, visit here. If you like the feed don’t forget to share it with your folks.
Hexagonal Architecture – Using Ports & Adapters to Make the Software Loosely Coupled & Easier to Test
A couple of days back I came across this term the Hexagonal Architecture. Had no clue what it was, did some reading. Eventually, sharing it with you folks.
The hexagonal architecture consists of three components Ports, Adapters & the Domain.
The focus of this architecture is to make different components of the application, independent, loosely coupled, easy to test.
The application should be designed in a way such that it can be tested by humans, automated tests, with mock databases, mock middleware, with & without a UI, without making any changes to the code.
The architectural pattern holds the domain at its core, that’s the business logic. On the outside, the outer layer has Ports & Adapters.
Ports act like an API, as an interface. All the input to the app goes through the interface. So, the external entities don’t have any direct interaction with the domain, the business logic.
The adapter is the implementations of the interface. They convert the data obtained from the ports, to be processed by the business logic. The business logic lies at the centre isolated & all the input and output is at the edges of the structure.
The Hexagonal shape of the structure doesn’t has anything to do with the pattern, it’s just a visual representation of the architecture.
Initially, the architecture was called as the Ports and the Adapter pattern, later the name Hexagonal stuck. The Ports & the Adapter analogy comes from the computer ports, as they act as the input interface to the external devices & the Adapter converts the signals obtained from the ports to be processed by the chips inside.
Should you wish to read more about the hexagonal architecture this is a good resource.
Coming down to the real-world code implementation, isn’t that’s what we already do with the Layered Architecture approach? We have different layers in our applications, we have the Controller, then the Service Layer interface, Class implementations of the interface, the business logic goes in the Domain model, a bit in the Service, Business and the Repository classes.
Well, yeah. That’s right. First up, I would say that the Hexagonal approach is an evolution of the layered architecture. It’s not entirely different.
As long as the business logic stays in one place, things should be fine. The issue with the layered approach is, often large repos end up with too many layers besides the regular service, repo & business ones.
The business logic gets scattered across the layers making testing, refactoring & pluggability of new entities difficult. Remember the stored procedures in the databases & the business logic coupled with the UI with JSP Java Server Pages?
When using JSPs and stored procedures, we still have the layered architecture, the UI layer is separate, the persistence layer is separate but the business logic is tightly coupled with the layers.
On the contrary, the Hexagonal pattern has its stance pretty clear, there is an inside component which holds the business logic & then the outside layer, the Ports & the Adapters which involve the databases, message queues, APIs & stuff.
Here is another project on GitHub on hexagonal architecture.
Mob programming is a programming approach where the entire team works on the same thing, at the same time at the same place, on the same computer.
Well, this is interesting, it’s like a head-on approach towards speeding up with the execution of super boring tasks assigned to a poor solo developer which he doesn’t want to do.
I can imagine everyone standing and staring at the boring tedious task with lasers coming out of their eyes. And the molecules of the super boring task start to disintegrate until the task completely disappears. Mob programming sounds fun.
Remote Teams Are On – Distributed Teams Help Startups Move Faster
If you have noticed lately, the number of job openings for remote programmers have a continual rise.
Primarily because it’s a win-win scenario for everyone. Developers don’t have to get stuck in traffic, they can work with the comfort of the homes. They are comparatively more productive.
The distributed communication tools are doing an amazing job of connecting people across the world, living in different time-zones.
StackOverflow believes in remote work & is pretty successful with it.
Airbnb & Udacity Abandon React-Native for Native Tech – Native Vs Hybrid Apps
Airbnb engineering in a series of blog posts share their experience of writing mobile apps with React-Native.
Native apps are preferred by businesses simply because of their performance. Also, they have full access to the device hardware, native APIs, the interface is fluid, the animations render pretty smooth.
A few downsides when writing native apps is, we have to maintain separate codebases for every platform. A separate codebase for IOS, Android, Windows etc. Different platforms different apps. This process is expensive for obvious reasons. Also, the native tech has a learning curve.
Hybrid apps are not quite as smooth as native apps in terms of performance & rendering & having access to the native APIs of the hardware. If you are writing a game, you are going to have a tough time achieving the desired performance with a hybrid app.
Since the Airbnb engineering team built their website using React, they considered React-Native as an opportunity to speed up the app development process by having one solo repo as opposed to having dedicated platform codebases.
They spent a couple of years working on it & eventually abandoned React-Native for the native technology. They shared insights into the pros & cons of writing a hybrid mobile app.
Ditching React-native does not mean it’s not powerful enough for writing mobile apps. It’s just that it did not fit a few of the required use cases.
If the app requirements are simple, React-native is a pretty powerful tech which could save us a lot of time & money.
The dev team at Airbnb was able to write features for cross-platform with upto 95 to 100% of the shared code & 0.2% of the platform-specific code. They used Redux for state management.
The primary issues when working with React-Native were that the framework isn’t as mature as the Native tech Android, IOS. They had to write a lot of patches to the React-Native library to get the desired functionality.
They found some of the trivial stuff when could be easily done with native tech, quite difficult to pull off with React-Native.
Here is another instance, where the Udacity engineering team abandoned React-Native due to the increased number of Android-specific features requests by the users.
Their android team was reluctant to go ahead with the hybrid app approach, the long-term maintenance costs of the React-Native codebase were high. They also faced UX consistency issues across the platforms.
In the past, Facebook has too invested quite some efforts into writing their mobile app with Html5 instead of the native tech & later admitted that it was a mistake.
Hybrid Vs Native What to Pick?
So, I think I am qualified to give an opinion on this. First thing, I would say is this is a trade-off. Also, it depends on the use case to some extent.
We just can’t discard hybrid tech on the grounds of performance and the availability of the other native features. If we have simple application requirements, for instance, a news app, hybrid tech is gonna save us a lot of time and money.
For use cases such as gaming apps, finance, fancy social media apps with animations & stuff, apps which have a lot of features & demand some serious computational power from the hardware. I would always pick the native tech stack.
Replicating Bugs in Distributed Systems, Microservices – Distributed Tracing
It’s kind of manageable to replicate bugs and fix them when working with solo monolithic codebases. But what if a feature is powered by a distributed system with several different microservices running on the backend?
In this scenario, looking into one microservice does not give the complete picture. Fixing stuff across microservices is one of the hardest things to pull off.
Distributed tracing is a mechanism which helps us trace and fix stuff in complex microservice environments. It helps us track requests, timing & the metadata of the data flow throughout the system.
I once had to fix an intermittent authentication issue across two microservices. Data from one flowed to the other, the second microservices often failed to authenticate the information incoming from the first microservice.
I spent days just trying to get to the bottom of the issue. It was hard, going through logs and putting debugs everywhere.
Finally tracked the issue, it was with the threads, sometimes the auth info got copied on to a local thread, sometimes it didn’t. The code was in Java, it used the Executor framework. I had to tweak the thread code to fix the issue. A distributed tracing framework might have come handy in that situation.
If you are intrigued further, you can look into Zipkin, it’s an open source distributed tracing system.
Looking For a Fast Front-end Programming Language? Checkout Elm
Why even think about Elm when we already have React, Angular?
Elm is fast. It focuses on simplicity & ease of use. There are no null, object undefined, NaN, Uncaught TypeError errors rather the errors are readable, developer friendly.
Elm enforces a Model View Update architecture. It’s a well-architected code, stays well-architected even when the app grows.
Uber’s visual insights team delivers a talk on Elm
What On Earth is a Kata Container?
We are all aware of the containers. Now, what is a Kata Container?
Well, it’s a secure container runtime with lightweight virtual machines which are just like containers but more secure. They leverage the hardware virtualization tech to isolate workloads.
The present containers have software based isolation they share the hardware. Kata containers provide the isolation on the hardware level too making things more secure. Here you go if you want to read more about it.
The name intrigued me pretty much when I heard it for the first time, thought will share it with you guys.
Developer Friendly Proxy Cloud Platforms
Cloud has become the norm for application deployment, it’s the preferred choice for everyone be it an indie developer, a startup or an enterprise. And this is for obvious reasons. Read Why Use Cloud? How is it Different from Traditional Computing?
But often we tend to get overwhelmed by the sheer number of services & options offered by the cloud giants in the industry. It can be intimidating sometimes for a developer with not so much DevOps experience who just simply want to deploy his code & be happy.
We as developers just want to focus on deploying our code & hitting the url in the browser instead of investing time in doing the research about the right service, infrastructure etc.
This led to the emergence of a new type of cloud platforms I would call them the developer-friendly proxy cloud platforms which further simplify the development process. They are more developer friendly, easy to use, simple. Ease of use is the USP of these platforms.
Here is good discussion on StackOverflow on deploying apps on Heroku vs AWS.
Besides the ease of use, these proxy cloud platforms are more dedicated towards specific use cases. Their pricing plans are more customized toward those use cases.
Netlify focuses on providing hosting & serverless backend for static websites.
Zeit is a dedicated serverless cloud platform as a service.
A good use case where I think these dedicated platforms seem fitting is when we are starting up either solo or in a small team & need to focus on building features, getting the MVP out, keep evolving it in fast iterations, & not worry about the infrastructure, investing time in research.
In the starting phases, a business can hardly afford a DevOps team.
Interested in Saving Birds?
By any chance, if you are interested in saving birds here is a really cool open source project called The Cacophony Project.
The project keeps a track on the bird predators like rats, possums etc via machine learning algorithms, thermal imaging, monitor the bird’s songs to measure the project impact in an area.
It is built using embedded systems, a web-based backend, machine learning pipeline. The technologies include Raspberry Pi 3, Node JS, PostgreSQL, Minio – an object store for AI, the test code is written in Python.
Building a Scalable Search Architecture
For basic search use cases, our old school SQL databases suffice but for some additional search feature requirements, we have to look up to technologies dedicated & built for running scalable search queries such as Elasticsearch & Solr.
Elasticsearch has become the defacto search framework of the industry considering the number of features it offers out of the box.
When using Elasticsearch a typical design is to stream in the search-related data to the Elasticsearch document-oriented database along with persisting the data in the database.
But there is a catch in this, what if during the persistence transaction involving two different distributed databases the write fails in one? How would you handle the distributed transaction failure?
Here is a nice write-up by confluent designing a scalable search architecture with Apache Kafka & distributed data stores. Kafka acts as the middleware data pipeline between the systems facilitating smooth data flow by decoupling the systems and enabling them to run independently.
REST APIs Vs Streaming APIs
REST APIs are fundamental to the large scale system design and web architecture today. Streaming APIs enable the continuous streaming of data to the client over HTTP. Typical examples of this is a video streaming service, streaming text in a microblogging site like Twitter, an internet radio service, online massive multiplayer browser-based games etc.
How different are the streaming APIs to the REST APIs? Is the difference subtle or are they architecturally different? Let’s find out.
To implement a data stream between the client and the server we have to establish a persistent connection between them. This is made possible by technologies such as a Message Queue, an in-memory cache on the backend, & on the client with Web Sockets, Comet, Server-sent events, Long Polling, Html5 Event Source API, Stomp etc.
A REST API is stateless, it’s more like a conversation between the client & the server. Every time the client sends all the information needed to get the response from the server. APIs don’t save any state.
On the contrary, the Streaming APIs having a continual open connection with the client. Instead of the client polling for data every time, the request is sent for the first time by the client & the server pushes the data to the client whenever there are new updates available.
The connection between the client and the server stays open with the help of heartbeat interceptors. These are just blank requests response between the client and the server to prevent the browser from killing the connection.
Twitter uses a streaming API to push the data to the client.
Cold Start Problem With Serverless – a Trade-Off
Serverless has gained quite some popularity over time. The FaaS Functions as a Service offering is saving businesses quite some money, especially if your application is stateless. For instance a tax calculation function or an image optimizing function which has nothing to do with the state of the previous request. Just feed the input and get the response.
Serverless functions save money solely due to the reason that businesses are charged only for the computation, not for the allocated instances. As there are no instances reserved to businesses. The micro-containers are assigned dynamically. The provisioning is automatic.
This brings along an issue with it, the issue of Cold Start.
If you are not aware. Cold Start simply means the additional time the cloud platform takes in warming up instances for the request. This happens if the function hasn’t been hit for a while.
Though the warm-up time isn’t too much still the platform has to load the code from the persistent disk, spin up a container, load the code in memory and execute the code.
This might be ok for some businesses that can afford this delay but also not ok for others running mission-critical tasks.
A workaround is to either make an external monitoring service responsible for not letting the instances sleep or writing a CRON job in the code which runs periodically at stipulated intervals to keep the instances warm.
The time after which idle instances go cold depends on the default configuration of the cloud provider also on the configurations set by the developer if any.
Cost Reduction & Open Security Strategies When Deploying Our Apps On the Cloud
Infoworld shares some insights on researching on third-party cloud security solutions besides the cloud-native security offered by the cloud platform for a more holistic & open security approach especially during multi-cloud deployments. Also, read about the upsides of buying reserved instances upfront in the cloud.
Cloud platforms today have marketplaces which offer ready to go solutions like technology stacks, services etc. which save businesses a lot of time. Averts the need for writing stuff from scratch or relying on the cloud platform for additional features.
Gitlab Ends Support for MySQL
Gitlab finally ended the support for MySQL due to several technical reasons. They used PostgreSQL & MySQL as their primary database. Majority of their customers were on PostgreSQL & a very small no was on MySQL.
The engineering team was facing problems making both the databases work together in conjunction. For instance, they wanted to use PostgreSQL’s Lateral Joins for optimizing the dashboard events but they couldn’t since this feature wasn’t available in MySQL.
Also, there was a lot of MySQL specific code which they had to manage which slowed them down.
[Industry] 75 million old jobs will go away by 2022, and 135 million new jobs will be created because of new technologies
Says Infosys president in a conversation with [email protected]. Tech has taken over there is no denying that it’s evolving at an unprecedented pace & the demand for software programmers in the industry is like never before.
And in order to not become a dinosaur, we have to be lifelong learners. Also, machines will take over the repetitive work & we will be more involved in doing the non-repetitive work. Which is pretty good if you ask me 🙂
PANE A Functional Programming Language with Visible Data
PANE is a live, functional programming environment built around data-visibility.
In PANE, all the intermediate values are visible by default, and we construct a program by acting on these concrete values. The language is built for general data processing.
Here is my understanding, In a PANE program at any stage the data would visible to the programmers by default just like we access it via the debug mode in other languages like Java or so. This is also known as the Principle of Radical Visibility.
The upsides of data visibility are, One – we can write faster efficient code since we are kinda aware right at the time when writing code that how the data flows through. And we can tweak our programs accordingly.
Second, let’s say we have an array of arrays or some input & we are not sure about what function to write to get the desired output. We provide PANE with the input and the desired output. PANE can figure out an apt function for us. This style is known as programming by example. You can run the demo here
Go Rises in Popularity – Especially When Implementing Concurrent Use Cases
Go programming language has a continual rise in the programming language popularity rankings of late. Primarily due to being a fitting tech when writing distributed systems.
Concurrency is a key thing when building distributed systems & Go really shines in implementing concurrent scenarios having an in-built concurrency mechanism.
I’ll just give a little background on Go. The language was built by Google to address some of the software infrastructural problems they were facing. They were having a hard time dealing with multicore processors, large scale computation clusters, infrastructural networking.
The code build time continually soared even on massive computation clusters.
Go was written to deal with all these scenarios and to make the development and production environment more productive.
It has an in-built concurrency, garbage collection & features to handle memory leaks which are critical to today’s software development landscape.
According to Google, Go is much better than C++ & Java in handling concurrency scenarios.
Also, the developers of Go were pretty motivated by this when writing the programming language.
Here is a good discussion on Go vs C++ on Reddit
Here is another Reddit discussion on using Go for writing distributed systems.
Some of the popular products written using Go are Kubernetes, Google uses Go on the Youtube backend & other services, Cockroach DB a strongly consistent horizontally scalable SQL database is written using Go, Bitcoin Lightning Network, Ethereum, Docker, Mattermost an open source alternative of Slack etc.
Having said this, I also want to state that there is no silver bullet, no perfect language, do your research having your use case in mind before picking up any language when building your product.
Concurrency Vs Parallelism
Now that I’ve been talking about Go, a programming language built for concurrent use cases. This is the perfect time to talk about Concurrency vs Parallelism.
What do you think is the difference between the two terms? Or there isn’t any? Or is it subtle?
When I first came across the two terms, I thought about it a bit. Hmmm…. Well, nothing came to my mind. I then re-thought about it in the context of web applications.
Concurrency is handling multiple users at one point in time by a web server whereas parallelism is executing the commands made by those concurrent users. Think of Ajax, running parallel user requests. Whereas concurrency is spawning separate threads by the server for every concurrent request.
If you want to delve more into concurrent & parallel programming languages here is a list of them on Wikipedia
Facebook Engineering Team Open Sources StateService: A State Machine as a Service for Dev Ops Teams
All of the services at Facebook are designed to be resilient. They are built intelligent enough to recover automatically from network outages.
In case they lose connection with other services running concurrently in real time. The entire service as a whole doesn’t get affected by a few services, or a data centre going down.
All of the production services at Facebook are built in-house. They also integrate with several other third-party services which run in VMs. Bouncing back from a third-party service outage isn’t such a simple task & requires time to get things back to normal.
To tackle this Facebook wrote a solution StateService which helps reduce the disaster recovery time. Facebook eventually open sourced the service to the community that would help the solution grow even better. Here is the GitHub repo.
Assess the Microservices Architecture of Your Business, Get a Best Practices Insight into the Individual Services Running
Chris Richardson, the creator of Cloudfoundry.com published a Microservices architecture assessment based on his years of experience running online systems, also training & consulting various organizations around the globe, helping them scale.
The assessment helps us compare against & improve the architecture of the microservices running in our production environments. The assessment helps us contemplate on our application architecture, organization & process. It evaluates the current architecture, figures out weaknesses & help fix it.
An Insight into the CERN Labs Open Data Portal Backend Technology Stack
In a SuperUser article, the technology lead behind the CERN Open Data portal talks about the backend technology stack of the portal. The portal holds petabytes of research data on particle physics & related stuff.
The Invenio framework is an open source framework for managing large-scale digital repositories.
Redis is used as a caching solution & the database is SQL based.
[Industry Trends] How Integration Of Blockchain With Gaming is Helping Gamers, Developers & Publishers
Implementation of blockchain & use of crypto tokens in games is gaining ground & this is not a bubble or a hype but for obvious reasons. Enabled by blockchain, gamers can carry around stuff, that is, items bought in one game to other games.
This averts the value of the items going null when or if a particular game goes out of business. Players can easily use the items bought in that game in other games.
You might have heard the term Multiverse in context to this. Multiverse simply means using items, bought in one gaming universe, in different gaming universes.
This also helps the businesses as users are not wary of the game being put offline by the game publisher & all their investments going void. They are more inclined to make the in-game microtransactions having the safety net of the blockchain.
[Career] Most In-Demand Skills Of 2019 by LinkedIn
Linked-In lists out the most in-demand skills of 2019. With Cloud computing, AI, UX design, Game Development, Animation, NLP & many more among them.
Code Your Way to A Smart Home with Raspberry Pi & Mozilla’s Things Getaway
This is one of the coolest things I’ve come across on the web. With Mozilla’s Things Getaway & Raspberry Pi we can control the devices on our home directly via the web from our phones or laptops.
Project Things by Mozilla is a framework of software and services that can bridge the communication gap between connected devices by giving “things” URLs on the web. Instead of having separate apps for controlling different devices. All the devices can be controlled from a single app.
Project Things focuses on three components:
The Things Gateway: This is an open source implementation that acts as a gateway. It bridges the IoT devices to the web.
Things Cloud – This is a collection of cloud services hosted by Mozilla. It helps with the management of quite a big number of IoT devices spread of a wide geographic area.
Things Framework – Things framework helps developers write modular, reusable components which further helps in creating IoT devices connecting directly to the Web.
In case you want to contribute to the project. The code is available on GitHub.
More On the Blog
- Latest Software Technology, Programming News, New Programming Languages Feed Vol.2
- Latest Software Technology, Programming News, New Programming Languages Feed Index
- Distributed Data Processing 101 – The Only Guide You’ll Ever Need
- What is Lift & Shift Migration to the Cloud? – An In-Depth Insight
- Distributed Cache 101 – The Only Guide You’ll Ever Need