Location: Phoenix Arizona Area
- Web: Apache, NGINX, Load Balancing, WAF, GSLB
- Infrastructure: DNS, Mail, SMTP, LDAP, TCP/IP, Routing, VPN, OAUTH2
- Virtualization: VMWare, Openstack, Openshift, AWS, Azure, GCP, SoftLayer
- Platform Automation: Cloud Foundry, BOSH, Kubernetes, Docker, Chef, Puppet, Ansible, Jenkins, Concourse, Terraform
- Software Development: Python, Ruby, C++, Go
- Storage: HDFS, Ceph, S3, Minio, Gluster, Elasticsearch
- Quote: “If you have to do the task regularly, put time into automating it.”
(10/2018-Current) (800+ Employees)
Portland Oregon - Remote
Senior Cloud Platform Architect
- Designed, wrote, and implemented the first One-Cloud Platform (1CP) for Smarsh’s Archiving Products, allowing for 1:1 deployment across AWS, Azure, IBM Softlayer, VSphere on-prem, and AWS Govcloud, running 99% Linux (over 10000), 1% Windows Server 2019 (less than 40).
- Designed our govcloud-based 1cp platform with FedRamp Authorization in mind. Smarsh’s Fedramp Authorization is still in progress.
- Implemented monitoring of all deployments and applications in both CloudFoundry and BOSH, allowing our PaaS team to maintain tens of thousands of virtual machines across dozens of different foundations almost completely with automation.
- Began work on automation around the migration from Cloud Foundry’s BOSH to Kubernetes Helm Charts, including data.
- Worked on multiple app transformation (AppTX) projects, as 12-factor app and infrastructure SME.
- vault-auto-unlocker boshrelease: Simple bosh release which automatically unlocks a vault deployment after start. Solves an automation problem
- “bastion-host” boshrelease: Dynamically generates temporary user access, with a persistent backing disk, allowing for credential sharing to never have to happen. When a user logs in, they’re given fresh credentials each time. Upon logout or session termination, those credentials are destroyed. While modular enough to use to target anything with a decent authentication api, we currently target the following:
- Cloud Foundry
- 1CP Pipelines: Originally started as a 20% project, but based off the theory that the same software should be able to function regardless of any backing IaaS, worked on standardizing the environment to meet the requirements of the application teams. Once those were standardized, most of the heavy lifting is handled by Concourse pipelines tailored for their respective IaaSes. Cuts deployment time down from weeks to hours.
- Object Storage
- Backing database (SQL/NoSQL)
- Logging Refactor: Refactoring the logging coming out of all vms and platforms to use
fluentd -> logstash -> elasticsearchas a logical network path. This allows more fine-tuned controls of the logs, and allows us to drop data at the source that is unnecessary for RCA. From there it drops straight to warm storage where the Log Polling project picks up the rest. The ELK cluster is custom bosh releases based off the bosh-elastic-stack project.
- Log Polling: Instead of capturing all logs, we capture a percentage of normal logs, and all Error logs. From there, use the data to do RCA. This cuts down on the amount of “hot” data in our largest ELK cluster over 30 days.
- ea-smtp-storage: Refactored our smtp-storage app based off requirements of high volume of mail being sent from a new customer. It was rewritten in Go, and based off Go-Guerrilla. Simple reception of email, quick storage to s3 for future processing.
- Presto Standalone boshrelease: Designed and created a bosh release of Presto using the standalone hive metastore that can scale to hundreds of worker nodes with minimal effort. Used as a standardized SQL interface for storage, such as s3. Useful when searching and indexing against petabytes of data in an object store, such as s3, minio, or azure blob storage.
CSAA Insurance Group
(7/16-2/2019) (3800+ Employees)
Phoenix Arizona - Remote
IT Platform Architect
- Designed and implemented Pivotal Cloud Foundry (PCF) platform for production use in AWS and local VSphere Clusters
- Wrote custom BOSH deployments for controlling Concourse CI, Elasticsearch, nginx, and maintenance tools.
- Wrote custom service brokers for on-demand instances of Elasticsearch, Postgres, and Mysql
- Automated user management and space security and configuration inside PCF
- Developed multiple POCs around platform automation, multi-stack configuration, and synchronization.
- Spearheaded multiple development and architecture platform projects around PCF implementation, including Kubernetes, Containerization, and automation.
- “NGINX Proxy” Release: Automated the deployment of NGINX for the use as reverse proxies for the platform. Also set these proxies up to be NAT load balancers to allow full isolation of the backend environment.
(8/15-1/18) (100+ Employees)
Shrewsbury, NJ - Remote
DevOps Lead Consultant
Primary Contract: Acxiom (4000+ Employees)
- Worked on chef cookbooks for everything from Base RHEL and Windows/IIS installs, to Cloudera Hadoop, RevR, JBoss, Email, MySQL, Windows/IIS, and Secure Data implementations.
- Wrote a custom connector to flow data from legacy monitoring systems into Zenoss, allowing for faster implementation.
- Wrote extensive integration and functional tests on both legacy and new cookbooks, using primarily ChefSpec, and ServerSpec.
- Automated RPM package generation using RPM Maven plugin.
- Developed a POC for compliance testing using the Chef Compliance suite, using custom parameters provided by Security team.
- Created POC Docker implementations using Kubernetes, Docker Swarm, and CoreOS's etcd/fleetd, and RHEL Atomic Host.
- Worked on POC VMWare VRA 7.0 Implementation.
General Electric Power and Water
(11/14-8/15) (300,000+ Employees)
Greenville, SC - Remote
Chef Specialist - Contract
- Created a Proof of Concept Chef development for bootstrapping servers in GE Power and Water's internal cloud environment.
- Worked closely with P&W operations portal development team to implement DevOps practices in their environment.
- Created Jenkins CI workflows for testing Chef Recipes.
- Wrote chef recipes for a multitude of automation projects.
- Implemented Vagrant for individual development testing.
- Wrote documents for defining coding standards and expectations for Automation projects and recipe writing, including setup, testing, development, naming conventions, and operational tasks.
- Worked closely with GE managers to implement SCRUM/Agile methodology for ticket tracking and development.
(04/14-08/14) (250 Employees)
Scottsdale, AZ - Onsite
IT Operations Manager
- Implemented a ticketing system for handling Operations requests.
- Standardized development habits.
- Worked directly with Developers and IT staff to migrate from a full-cloud to hybrid infrastructure using OpenStack.
- Implemented an in-house Datacenter using OpenStack with more than 30 nodes.
- Implemented a Galera-MySQL cluster to handle replication from in-house to cloud provider.
- Migrated primary virtualization technology from SmartOS to OpenStack.
- Created POC environment, testing Canonical, Mirantis, Dell, HP, and RedHat Openstack Implementations.
- Negotiated pricing with vendors.
- Researched and automated Neutron network implementations using Python scripts, allowing us to quickly bring up a new site with full VPN within hours instead of days.
- Slipstreamed the creation of new application and development servers, allowing for autoscaling both inside of OpenStack, but in our external cloud provider.
- Implemented the use of Docker for all software development, and production applications.
(01/14-04/14) (10,000+ Employees)
New York, NY - Remote
Contractor to ATOS - Senior Product Engineer - Product Engineering
- Wrote documentation and standardized the use of Apache 2.4 in the McGraw-Hill Financial environment.
- Championed the use of using the default RPMs, and adding any "custom" configuration in a separate package, installable via YUM or PKGINST.
- Created documentation on the proper use of AWS Cloud Products.
- Assisted the engineering teams with day-to-day issues they would have, answering questions about best practices in their applications.
Independent Contractor for Confidential Company
(09/13-12/13) (50+ Employees)
Mesa, AZ - Onsite
Director of IT and Engineering
- Implemented point-to-point vpn connectivity from local office to remote datacenter.
- Changed the application layer to be multi-tiered, allowing for more work to be done.
- Wrote a distributed CDN system, allowing for addition and removal of hosts on the fly, and multi-tiered distribution of files across multiple networks.
- Implemented a ticket tracking system for all software development projects.
- Scoped out and estimated applications.
- Moved all source code delivery from SVN to GIT
- Managed time tracking for 10 developers and 2 sysadmins.
- Managed budgetary expectations for the teams.
(08/12-08/13) (800+ Employees)
Scottsdale, AZ - Remote
Sr. Software Engineer - Development Infrastructure
- Created, designed, and implemented auto-scaling solutions using a 100% cloud-based server architecture using Windows (EC2), Linux (Amazon Web Services/Google Compute Engine), and Google App Engine (Python) platforms.
- Worked with the company's production server support team to help them fill gaps in their knowledge base with proper documentation.
- Assisted with training of developers on writing proper documentation and testing techniques, and Created run-book templates that are now the gold standard for application documentation for the company.
- Worked with local teams on release techniques and go-live procedures, assisting with "finding the loose ends" that could possibly be missed.
- Helped transition multiple enterprise applications from a push-based, single-server model, to a queue-based, autoscaling model.
- Wrote standardized "cookbooks" for deploying of applications, including methods of downloading and updating code.
- Worked with the IT team to define the “Separation of Duties” between IT and Development. Worked with the Director of IT, to standardize and define the roles for each employee in both IT and Development Infrastructure.
(08/11-08/12) (750+ Employees)
Scottsdale, AZ - Onsite
Sr. Systems Operations Engineer - Production Engineering and Operations
- Wrote documentation and run-books for off site operations.
- Designed and wrote deployment automation for multiple AWS instances, using puppet
- Set up clusters of non-standard server types, such as SQUID proxies, defining custom metrics for autoscaling on EC2 using CloudWatch.
- Migrated large quantities of data from MSSQL to MySQL (over 2TB)
- Designed and wrote a functional backup system for backing up and replicating over 20TB of data per week off-site without using tape and holding a total of over 250 TB of data.
- Rewrote operations dashboard, moving it from PHP to Python, while both keeping and improving on it's functionality.
- Wrote multiple automated control scripts for doing many specialized jobs, such as finding bad AWS instances, or monitoring network load. Based off of those scripts and load, we could autoscale servers and systems that under normal circumstances, we would be unable to.
- Wrote documentation and specifications for an auto-scaling application inside of AWS that uses between 10 and 240 autoscaling instances, based off load.
(02/11-08/11) (50+ Employees)
Huntsville, AL - Onsite
Senior Systems Administrator
- Managed DNS, DHCP, Active Directory, IIS, SQL Server 2008, WSUS, and WDS images.
- Managed and created new MySQL, Postgres,Nginx web server, PHP, and Python servers using Puppet and Capistrano.
- Maintained revision and software control of entire environment using Mercurial and Git repositories.
- Monitored and maintained over 90 PHP-based fansites running MediaWiki, Vbulletin, and Invision Power Board.
- Monitored over 30 .NET 4.0 sites running IIS 7 running custom in-house code. All revision control and management was done via scripting automation using a combination of powershell and mercurial repositories.
- Managed all SAN and Storage for the environment, set up backups for all databases in the environment, and implemented a shared NAS for static media in our environment.
- Wrote and architected a plan to virtualize all the windows and linux servers, allowing us to maximize the use of our hardware, using Citrix XenServer and VMWare ESXi 4.1.
- Assisted with Python Development and MySQL performance optimization.
- Worked heavily with windows development team to streamline deploy processes. Assisted with resolving multiple issues involving SQL 2008 R2, and migration from IIS6 to IIS7 on our applications.
- Did CI Integration of the .NET platform Applications using Jenkins.
(11/07-02/11) (550+ Employees Nationwide, 35 Locally)
Phoenix, AZ - Onsite
Systems Administrator – AZ.Gov
- In charge of web maintenance and administration for approximately 80 servers, 100 virtual machines, running AIX, Linux (RHEL), VMWare vSphere, Eucalyptus, and Windows
- Created 24/7 On-call schedule between systems administrators
- Worked directly with software vendors to find best solution for a scalable infrastructure
- Designed a working, customized version of JBoss that is scalable and allows for clustering across multiple environments
- Worked with developers on setting up best practices for code maintenance, release scheduling, and design.
- Designed scalable HA web and file architecture that was Active-Active across 2 sites, allowing for Disaster recovery, including capacity planning.
- Set up a clustered file server based off of Redhat's GFS and clustering suites.
- Assisted in rewriting approximately 80 applications, which we were contractually obligated to, in approximately 2 years time. These applications included everything from simple licensing applications, to payment gateways, SSO solutions, making them all scalable and easily maintained
- Did CI Integration of the applications using Jenkins for both the JAVA and .NET environments.
(07/05-07/07) (300,000+ Employees)
Scottsdale, AZ - 60% Travel
Senior Systems Software Engineer - ASSD Team
- Engineered, and assisted with development of company's largest internal project to date.
- Provided support for the application environment 24/7.
- Maintained operational status for the project.
- Supported offshore teams for 24/7 support issues they had.
- Designed and wrote TAT testing plan for 3 automated hub systems that pushed approximately 1.5 million packages nightly.
- As the co-lead of the team, created a team of 8 people to support the automated sort system 24/7, with rotating shifts.
- Designed and wrote the standards for WebSphere 6.0 deployment for DHL.
- Created and designed debugging systems for finding and tracking package "messages" inside the system.
- Assisted with troubleshooting Allen-Bradley PLC Systems that were used for tilt tray control.
- Helped with 6-side scanner calibration.
- Worked directly with Siemens on implementation and testing of their tilt tray and introduction control systems.
- Worked between 2 very unique environments: one being the corporate environment back in Scottsdale, and the other being on-site at the hubs, which was both a construction zone, and a working sort facility at the same time.
- Travelled approximately 60% to spend time on-site at the sorting Hubs.
- Designed, wrote, and ran the TAT testing plan for 3 automated hub systems.
(01/05-03/05) (300+ Employees)
Scottsdale, AZ - onsite
Linux Systems Administrator - Tiger Team
- Maintained hundreds of web servers running Linux/Apache.
- Worked directly with customers on system problems, supplying customized support for the largest clients.
- Created knowledge base currently in process of possibly being implemented site-wide.
Senior Linux/UNIX Systems Administrator (IBM Contractor to American Express)
- Maintained multiple Application and web servers running Solaris and WebSphere running very distinct environments
- Installed updates and worked directly with Application teams at American Express to schedule change processes.
- Required adherence to Processes put in place and documentation of all activities on servers.
- Used Documentation systems for management of information.
- Was on 24/7 call, and regularly worked long hours
AOL Time Warner Corporation - Netscape
(06/01-01/03) (10,000+ Employees)
Mountain View, CA
Release Engineer - Netscape Client Product Division
- Installed, maintained, patched, upgraded 20 build machines specifically for development of Netscape products, and Mozilla servers.
- Assisted with build script maintenance for all client projects.
- Was Build/Release Engineer on Netscape 4.76-4.80, Netscape 6.1-7.0, Mozilla 0.7-1.3b
- Was on rotation for daily trunk release engineer, and in charge of making sure that all check-ins were authorized and would not break the tree.
Netscape Communications Corporation
(05/00-06/01) (1,000+ Employees)
Mountain View, CA
Systems Administrator - iPlanet Learning Solutions
- Maintained Automation for approximately 2400 training machines worldwide at iPlanet Learning.
- Helped with the design and implementation of the "Classroom in a box" Project, which allowed for us to set up, on the fly, temporary classrooms anywhere in the world.
- Assisted with the technical details and load-balancing for RealServers both internally and externally at iPlanet/Netscape.
- Created and Implemented Design Documents for Centralized Courseware installation, software updates, and Automation for approximately 2400 Solaris, and Windows NT Machines Worldwide.
- Developed Automation scripts for NT for installing complex software such as Oracle, and iPlanet Server products.