Josiah Berkebile

I worked in a fast-paced, goal-oriented Ruby on Rails Agile web app development team at Healthcare IT company for 6-9 months, and then single-handedly maintained a large Ruby on Rails app for a large financial institution for about 2 years. I've also done a few hackathons that involved building a web app for a non-profit in less than 24 hours. For 2 of the 3 hackathons, I organized and lead the team. Here's the end result of some of those challenges: http://principalsconnect.com/ http://adoption.kvc.org/

I have 8 years of extensive experience writing Java applications that run in a Hadoop environment. I've developed data pipelines in Apache Crunch, created custom Flume Clients, and also Flume Sources that plugin to the running Flume Agents per the Flume API, and written MapReduce algorithms that process data in the petabytes scale. Some of these Java applications ran as services and used Jersey to expose a web API and Solr to index data sitting in NoSQL storage. Spring is a bit of a heavy-weight. As a result, the various Hadoop engineering teams I've been on have chosen Dagger, rather than Spring, as the dependency injection framework for our applications since it's faster and more lightweight.

I have six (6) years of experience using Scala in Hadoop and data engineering environments. I've written a LOT of Scala. Some of the Flume plugins I've written have been written in Scala. I've written a Flume Client that used Scalatra to listen for events using Webhooks and then distributed them to an array of Flume Clients using the Akka concurrency framework. I've extended and refactored a Scala Akka web scraper. I've leveraged ScalaTest and ScalaCheck in practically all of my Scala projects. In my projects as a Scala consultant a year ago I've leveraged the Typelevel stack to develop streaming applications using a pure functional approach. I've also written a few Spark applications in Scala. Some of my smaller projects have involved implementing a PageRank and Collaborative Filtering algorithms, and the most ambitious project I've done on Spark was an NLP classifier. I have a decent amount of experience architecting, profiling, and tuning Spark applications. All of the applications I've written in Scala have been highly concurrent, some of them have leveraged Software Transactional Memory libraries like ScalaSTM to simplify multi-threaded interactions with shared memory.

Python is a versatile language, and my experience in it is also very versatile. I've used Python to do data exploration using NumPy and Pandas, I've taken Python code from data scientists and translated it into PySpark or into Scala Spark applications, I've written some systems automation scripts in Python, and written scripts for moving or ingesting data between databases and systems on the petabytes scale. I have about equal experience using legacy Python 2 as well as Python 3 in production environments.

I used Haskell as my vehicle for learning Functional Programming. Since Haskell is a pure functional language, it would not let me fall back on my old procedural object-oriented habits. I succeeded in learning enough Haskell on my own to become productive in the language. This learning enabled me to pick-up Scala more easily than most other members of the engineering teams I've been a part of. If you are learning Functional Programming or just getting started with Haskell, I can certainly get you to a level where you will be productive in the language.