Projects

DATA ENGINEERING

Airflow Pipeline for PostgreSQL-BigQuery

Simple Airflow ELT Pipeline for PostgreSQL-BigQuery

Apache Airflow

https://medium.com/@dmitri.mahayana/simple-airflow-pipeline-for-postgresql-bigquery-5f8dfcbbe1cd

DBT Snowflake Model

Design and Build DBT for Snowflake Model

What is dbt?

https://medium.com/@dmitri.mahayana/design-and-build-dbt-for-snowflake-model-ca689ce9e3e3

IDX Stock App: Real-Time Data Streaming with Kafka Raft

IDX Stock — Real Time Data Streaming with Kafka Raft

Apache Kafka

https://medium.com/@dmitri.mahayana/idx-stock-real-data-streaming-with-kafka-97c479cff1b9

Real-Time Data Processing with Flink and Kafka

Consuming Kafka Topics using Python Flink Table API

Today’s consumers have come to expect timely and accurate information from the companies they do business with. Whether it’s being alerted…

https://medium.com/@dmitri.mahayana/consuming-kafka-topics-using-python-flink-table-api-6982d63ee9bd

IDX Stock App: Real-Time Stock Prediction with Flink Machine Learning

IDX Stock — Real Time Data Prediction with Flink-Kafka-MongoDB

This article is a continuation of the IDX Stock Application that we built using Kafka. Earlier, we built the producer and ksql consumer to…

https://medium.com/@dmitri.mahayana/idx-stock-real-time-data-prediction-with-flink-kafka-mongodb-526c6abf291f

Multi Threading Web Crawler

Web Scrapping with Playwright

Web Scraping Demo Video

https://medium.com/@dmitri.mahayana/web-scrapping-with-playwright-f6ce9fad2b49

DATA SCIENCE & AI

Chain Messages with LangGraph

Chain Everything with LangGraph

As AI continues to transform how we work, interact, and solve complex problems, tools that enhance our ability to leverage language models…

https://medium.com/@dmitri.mahayana/chain-everything-with-langgraph-9a0f35b2d7a2

Haystack Framework 2.x

Migration Haystack Framework 2.x

Haystack 2.x is a major update to the design of Haystack components, Document Stores, and pipelines. Based on the online doc, they believe…

https://medium.com/@dmitri.mahayana/migration-haystack-framework-2-x-f58b5c585d61

ChatGPT using Python-LangChain

ChatGPT Template using Python-LangChain

Customize ChatGPT Prompt

https://medium.com/@dmitri.mahayana/chatgpt-template-using-python-langchain-c201de474122

Creating Virtual Assistance using with Llama-2 7B Chat Model

Creating Virtual Assistance using with Llama-2 7B Chat Model

I know many of us are already hands-on using OpenAI to build virtual assistance, but do know if Meta has released their own LLM to compete…

https://medium.com/@dmitri.mahayana/creating-virtual-assistance-using-with-llama-2-7b-chat-model-9f693c8250ee

Chatbot Text to Image using Stable Diffusion and Streamlit

Create Chatbot Text to Image using Stable Diffusion and Streamlit

You can create images from text input in a few different ways, but have you tried Stable Diffusion?

https://medium.com/@dmitri.mahayana/create-chatbot-text-to-image-using-stable-diffusion-and-streamlit-fb88ffda97d2

Build Machine Learning and Deploy to Streamlit Cloud

Build Machine Learning and Deploy to Streamlit Cloud

Are you looking for other open-source visualization apart from Matplotlib, Tableau, or Power BI?

https://medium.com/@dmitri.mahayana/build-machine-learning-and-deploy-to-streamlit-cloud-634cad9b6c71

Training Our Dataset for Llama2 with RAG LangChain

Training Your Own Dataset in Llama2 using RAG LangChain

RAG is a technique for augmenting LLM knowledge with additional, often private or real-time, data. LLMs can reason about wide-ranging…

https://medium.com/@dmitri.mahayana/training-your-own-dataset-in-llama2-using-rag-langchain-e4991064f423

Vertex AI and Python for Bounding Box Dataset

Vertex AI and Python for Bounding Box Object Dataset

Vertex AI offers everything you need to build and use generative AI — from AI solutions, to Search and Conversation, to 130+ foundation…

https://medium.com/@dmitri.mahayana/vertex-ai-and-python-for-bounding-box-object-dataset-b817a88e0551

Ultimate Semantic Search with Haystack Framework

Ultimate Semantics Search — Haystack Framework — Part 1

Haystack emerges as a cutting-edge open-source framework designed to seamlessly integrate large language models (LLMs) into…

https://medium.com/@dmitri.mahayana/ultimate-semantics-search-haystack-framework-part-1-4e3ba5bee3df

Metadata Filtering using Custom Node in Haystack Framework

Ultimate Semantics Search — Part 2 — Metadata Filtering

Although new retrieval techniques are great, sometimes you just know that you want to perform search on a specific group of documents in…

https://medium.com/@dmitri.mahayana/ultimate-semantics-search-part-2-metadata-filtering-05cad97bc5da

More Projects via GitHub

dmitrimahayana - Overview

Freelancer | Big Data Engineer | Data Engineer | Cyclist - dmitrimahayana

https://github.com/dmitrimahayana