Arthur J.

Data Engineer

Data Engineer
11/1/2020 - 8/1/2022

Achieved expertise in data migration within the Azure environment utilizing Data Factory. Demonstrated advanced skills in process authorization through Python scripting. Developed and managed robust data pipelines using Spark in Databricks. Successfully deployed machine learning models into production environments, ensuring operational efficiency and accuracy.

Data Engineer
7/1/2019 - 10/1/2020

Gained expertise in managing and maintaining Hadoop environments, focusing on optimizing performance and ensuring system reliability. Demonstrated proficiency with Hadoop Distributed File System (HDFS), MapReduce, and YARN resource management. Built and maintained robust data pipelines using Apache Hive, Pig, and HBase, ensuring smooth data flow and processing. Showcased skills in monitoring and debugging Hadoop clusters using tools like Apache Ambari and Cloudera Manager. Gained hands-on experience in various big data ecosystem components, including Apache Spark, Kafka, and Flume, to enhance data ingestion and transformation processes. Enabled data analysis and business intelligence efforts by integrating Hadoop-based analytics with visualization tools such as Tableau and Power BI.

Data Engineer
12/2/2016 - Present

Developed expertise in data analysis of both structured and unstructured data using Cloudera's Apache Hive and Impala distributions. Managed and processed data within the Hadoop environment. Created process automation scripts utilizing Python and shell-script. Contributed to the development of IoT projects and implemented batch and real-time data processing solutions with Apache Spark, Kafka, and Elasticsearch.

IT Analyst
1/2/2015 - 9/2/2015

Developed proficiency in migrating versioned projects from SVN to Git. Standardized project versioning processes through automation utilizing Python and Jenkins. Created and maintained automated screen tests employing Selenium, Python, and Testlink. Enhanced automated monitoring systems incorporating Python, Selenium, and Zabbix.

Java Developer
2/2/2012 - 12/2/2014

Developed critical mission web systems tailored for the financial market utilizing Java, JBoss, Wicket, and Hibernate. Enhanced software reliability through comprehensive unit testing with JUnit and performed integration testing using proprietary software and Selenium. Ensured smooth development workflows by implementing continuous integration with Jenkins and adhered to agile methodologies to maintain project efficiency and adaptability.

Bachelor of Science and Technology at Federal University of ABC
2015 - 2019