In Part 1, we established the high-level roadmap of LangChain — exploring the architectural shift toward chat models and breaking down the foundational RAG pipeline.
Now, we dive directly into code implementation. Before assembling end-to-end chains or building multi-step agents, we must master two foundational pillars: integrating open-source models and structuring production prompt templates.
While proprietary APIs (like OpenAI or Anthropic) offer convenience, open-source models provide full data privacy, fine-tuning flexibility, and infrastructure control.
The largest central hub for discovering, testing, and down open-source LLMs is Hugging Face. When integrating these models into your LangChain stack, you have two primary options:
For implementing the below-given codes, you need to get HuggingFace API keys. After you get the API key, copy it and store it in a .env file in the project folder where you will be storing all your LangChain tutorial program files.
Please refer to the tutorial link on how to get HuggingFace API keys: https://www.geeksforgeeks.org/artificial-intelligence/how-to-access-huggingface-api-key/
Also, I have not provided the program outputs for this blog, since I want my readers to actually try the code snippets in their systems.
Here is how you connect both hosted endpoints and local inference pipelines inside LangChain:
import osfrom langchain_huggingface import HuggingFaceEndpoint, HuggingFacePipelinefrom transformers import AutoModelForCausalLM, AutoTokenizer, pipeline# 1. Using Hugging Face Inference API (Serverless / Dedicated Endpoint)os.environ["HUGGINGFACEHUB_API_TOKEN"] = "your_hf_api_token"endpoint_llm = HuggingFaceEndpoint( repo_id="mistralai/Mistral-7B-Instruct-v0.2", task="text-generation", max_new_tokens=256, temperature=0.2)# 2. Running Locally using Transformers Pipelinemodel_id = "meta-llama/Llama-2-7b-chat-hf"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")pipe = pipeline( "text-generation", model=model, tokenizer=tokenizer, max_new_tokens=256, temperature=0.3)local_llm = HuggingFacePipeline(pipeline=pipe)
A prompt is the input instruction or query provided to a language model to guide its output. In modern conversational architectures, prompts are categorized across distinct roles to establish context, personas, and system boundaries.
Note: We will focus on the LCEL and the formation of chains in langchain in detail in the upcoming blogs. In this blog, I only want you to focus on the prompt part only. You can run the code snippets for now if you wish to do so.
Unlike raw end-user prompts (which are typed directly by users) or system prompts (which define high-level personas and safety guardrails), a developer prompt programmatically blends application logic, formatting constraints, and task rules with dynamic user data before sending the payload to the LLM.
Implementation: Production-Grade Customer Support Triage
This implementation demonstrates a developer instruction pipeline that ingests a raw customer support ticket, extracts a concise 2-sentence summary, classifies the urgency into strict enum levels (LOW, MEDIUM & HIGH), and returns a validated Pydantic object
import osfrom langchain_core.prompts import PromptTemplatefrom langchain_core.output_parsers import StrOutputParserfrom langchain_openai import ChatOpenAI# 1. Initialize modelmodel = ChatOpenAI(model="gpt-4o-mini", temperature=0)# 2. Define developer instruction promptdeveloper_prompt = PromptTemplate.from_template( """Summarize the following customer support ticket in exactly 2 sentences.Flag the urgency level strictly as LOW, MEDIUM, or HIGH.Ticket Details:{ticket_text}Output Format:Summary: <summary>Urgency: <urgency>""")# 3. Create LCEL Chain (Prompt -> Model -> String Parser)chain = developer_prompt | model | StrOutputParser()# 4. Invoke with runtime inputticket_data = "User cannot log into dashboard after password reset. Multiple attempts failed and account locked."response = chain.invoke({"ticket_text": ticket_data})print(response)
Why Developer Prompts Matter in Production
2. Multi-Role Conversational Prompt:
Unlike single-turn base models that treat text as an unstructured continuous string, modern instruction-tuned Chat Models process structured message sequences.
A multi-role conversational prompt decomposes an interaction into explicit functional roles (System, User, Assistant), allowing you to steer identity, enforce security guardrails, and dynamically inject historical context without confusing the model.
Key Roles Breakdown
Implementation: Dynamic Multi-Turn Chat with State Management
In production, you rarely hardcode past turns. Instead, you use ChatPromptTemplate combined with MessagesPlaceholder and RunnableWithMessageHistory to manage multi-turn conversational state across unique user sessions:
from langchain_core.prompts import ChatPromptTemplatefrom langchain_core.output_parsers import StrOutputParserfrom langchain_openai import ChatOpenAImodel = ChatOpenAI(model="gpt-4o-mini", temperature=0.2)# Multi-turn chat template with role awarenesschat_prompt = ChatPromptTemplate.from_messages([ ("system", "You are a professional financial advisor for a retail bank. Only discuss bank products, never give speculative advice, and include a non-advice disclaimer."), ("user", "Can you help me understand what a Roth IRA is?"), ("assistant", "A Roth IRA is an individual retirement account where you contribute after-tax dollars, allowing tax-free growth and tax-free withdrawals in retirement. (Note: This is general information, not investment advice.)"), ("user", "{user_query}")])chain = chat_prompt | model | StrOutputParser()response = chain.invoke({"user_query": "Does our bank offer automated deposits for a Roth IRA?"})print(response)
When to Use Multi-Role Conversational Prompts
3. Zero-Shot Prompting
Zero-Shot Prompting is the simplest and most direct prompting technique in Generative AI. In a zero-shot setup, you present a task, query, or instruction to the language model without providing any prior examples or demonstration input-output pairs.
The model relies entirely on the pre-trained knowledge, linguistic patterns, and semantic understanding it acquired during training to perform the task on the first attempt.
Implementation: Zero-Shot Sentiment & Intent Classifier
Here is a complete, production-grade zero-shot implementation using langchain_core and ChatOpenAI with strict enum-based structured outputs to ensure deterministic classification:
from langchain_core.prompts import PromptTemplatefrom langchain_core.output_parsers import StrOutputParserfrom langchain_openai import ChatOpenAImodel = ChatOpenAI(model="gpt-4o-mini", temperature=0)zero_shot_prompt = PromptTemplate.from_template( """Classify the sentiment of the following customer review strictly as 'Positive', 'Negative', or 'Neutral'.Review: "{review}"Sentiment:""")chain = zero_shot_prompt | model | StrOutputParser()review_input = "The battery life is decent, but the camera quality disappointed me."result = chain.invoke({"review": review_input})print(result)
Key Characteristics
4. Few-Shot Prompting
Few-Shot Prompting is a technique where you supply explicit input-output demonstration pairs inside the prompt before presenting the actual query.
While zero-shot relies strictly on pre-trained internal patterns, few-shot prompting conditions the model in-context on the exact structure, nuance, formatting constraints, and domain edge cases you expect.
Implementation: Dynamic Few-Shot Sentiment & Sub-Score Extraction
LangChain provides dedicated primitives (FewShotChatMessagePromptTemplate and FewShotPromptTemplate) to assemble example pools programmatically:
from langchain_core.prompts import FewShotPromptTemplate, PromptTemplatefrom langchain_core.output_parsers import StrOutputParserfrom langchain_openai import ChatOpenAImodel = ChatOpenAI(model="gpt-4o-mini", temperature=0)# 1. Provide demonstration examplesexamples = [ {"review": "This product changed my life!", "sentiment": "Positive"}, {"review": "Broke after two days, total waste of money.", "sentiment": "Negative"}]# 2. Template for each individual exampleexample_prompt = PromptTemplate.from_template("Review: \"{review}\"\nSentiment: {sentiment}")# 3. Assemble Few-Shot Templatefew_shot_prompt = FewShotPromptTemplate( examples=examples, example_prompt=example_prompt, prefix="Classify sentiment as Positive, Negative, or Neutral.\n", suffix="Review: \"{review}\"\nSentiment:", input_variables=["review"])chain = few_shot_prompt | model | StrOutputParser()result = chain.invoke({"review": "The battery life is decent, but the camera quality disappointed me."})print(result)
5. Chain-of-Thought (CoT) Prompting
Chain-of-thought prompting explicitly asks the model to reason step by step before answering. This can be helpful in solving mathematical problems.
from langchain_core.prompts import PromptTemplatefrom langchain_core.output_parsers import StrOutputParserfrom langchain_openai import ChatOpenAImodel = ChatOpenAI(model="gpt-4o-mini", temperature=0)cot_prompt = PromptTemplate.from_template( """Solve the following math problem. Think step-by-step and write out your reasoning before stating the final answer.Question: {math_problem}Reasoning:""")chain = cot_prompt | model | StrOutputParser()problem = "A store had 120 apples. They sold 45 in the morning and 30 in the afternoon. How many apples are left?"result = chain.invoke({"math_problem": problem})print(result)
6. Retrieval-Augmented (RAG) Grounding Prompt:
Restricts model answers strictly to injected reference passages.
from langchain_core.prompts import PromptTemplatefrom langchain_core.output_parsers import StrOutputParserfrom langchain_openai import ChatOpenAImodel = ChatOpenAI(model="gpt-4o-mini", temperature=0)rag_prompt = PromptTemplate.from_template( """You are a customer support agent. Answer the question using ONLY the provided context below. If the answer cannot be found in the context, output 'Information not found.'Context:{retrieved_context}Question: {user_question}Answer:""")chain = rag_prompt | model | StrOutputParser()context_data = "Our refund policy allows returns within 30 days of purchase, with a 15% restocking fee for opened electronics."query = "Can a customer return an opened laptop after 20 days?"result = chain.invoke({ "retrieved_context": context_data, "user_question": query})print(result)
7. Tree-of-Thought Prompting:
While Chain-of-Thought (CoT) forces a model to reason in a straight, linear path, it has a major weakness: if the model makes an incorrect assumption in step 1, that error cascades down the entire chain.
Tree-of-Thought (ToT) generalizes over CoT by allowing the model to explore a nonlinearD problem space. It treats reasoning like a search tree where the system generates multiple candidate ideas (branches), evaluates each branch against specific constraints, discards weak paths (pruning), and expands the most promising ones.
Implementation: Building a Tree-of-Thought Pipeline
Because true Tree-of-Thought requires branching, scoring, and decision-making, it is implemented at the orchestration layer using multi-step chains with structured validation.
from langchain_core.prompts import ChatPromptTemplatefrom langchain_core.output_parsers import StrOutputParserfrom langchain_openai import ChatOpenAIfrom pydantic import BaseModel, Field# Initialize Modelmodel = ChatOpenAI(model="gpt-4o-mini", temperature=0.7)# --- STEP 1: Branch Generation (Propose Candidate Thoughts) ---generate_branches_prompt = ChatPromptTemplate.from_template( """You are an expert itinerary planner. Generate exactly 3 distinctly different 3-day travel itinerary approaches for the following problem.Problem: {problem}Format output:Approach 1: <summary of focus and day-by-day plan>Approach 2: <summary of focus and day-by-day plan>Approach 3: <summary of focus and day-by-day plan>""")branch_generator_chain = generate_branches_prompt | model | StrOutputParser()# --- STEP 2: Evaluation & Scoring (Pydantic Structured Schema) ---class ItineraryEvaluation(BaseModel): chosen_approach_number: int = Field(description="The number (1, 2, or 3) of the best approach") reasoning: str = Field(description="Why this approach best satisfies budget, culture, and feasibility") score_out_of_10: float = Field(description="Feasibility and quality score out of 10")evaluator_prompt = ChatPromptTemplate.from_template( """Evaluate the following 3 itinerary options based on time feasibility, cultural balance, and budget efficiency.Candidate Options:{candidates}Select the single most viable candidate branch to expand.""")evaluator_model = model.with_structured_output(ItineraryEvaluation)evaluator_chain = evaluator_prompt | evaluator_model# --- STEP 3: Branch Expansion (Final Execution on Selected Node) ---expansion_prompt = ChatPromptTemplate.from_template( """Expand the selected travel strategy into a detailed, hourly execution plan including transit details and meal recommendations.Selected Strategy:{selected_plan}""")expansion_chain = expansion_prompt | model | StrOutputParser()# ==========================================# --- EXECUTION FLOW (Orchestrating ToT) ---# ==========================================problem_statement = "Plan a 3-day trip to Tokyo balancing historical culture, street food, and budget constraints."# 1. Generate Multiple Reasoning Branchesprint("Generating candidate thought branches...")candidates = branch_generator_chain.invoke({"problem": problem_statement})print(f"\n--- Generated Candidates ---\n{candidates}\n")# 2. Evaluate & Prune Branchesprint("Evaluating candidate branches...")evaluation = evaluator_chain.invoke({"candidates": candidates})print(f"\n--- Evaluation Verdict ---\nSelected Branch: Approach {evaluation.chosen_approach_number}")print(f"Score: {evaluation.score_out_of_10}/10\nReason: {evaluation.reasoning}\n")# 3. Expand the Winning Branchprint("Expanding winning branch into full itinerary...")final_itinerary = expansion_chain.invoke({"selected_plan": candidates})print(f"\n--- Final Optimized Output ---\n{final_itinerary}")
When to Use Tree-of-Thought
8. Function Calling / Tool-Use (Structured Output Schema)
Binds JSON schemas to the model so it can return structured tool arguments.
from langchain_openai import ChatOpenAI# 1. Define Tool Schematools = [ { "type": "function", "function": { "name": "get_stock_price", "description": "Fetch current stock price for a ticker symbol.", "parameters": { "type": "object", "properties": { "ticker": { "type": "string", "description": "Stock ticker symbol (e.g., AAPL, MSFT)" } }, "required": ["ticker"] } } }]# 2. Bind tool to Chat Modelmodel = ChatOpenAI(model="gpt-4o-mini", temperature=0)model_with_tools = model.bind_tools(tools)# 3. Model decides when to invoke toolresponse = model_with_tools.invoke("What is Apple's stock price right now?")print(response.tool_calls)# Output: [{'name': 'get_stock_price', 'args': {'ticker': 'AAPL'}, 'id': '...'}]
That is it for today’s blog. Stay tuned as more interesting blogs will come up building upon these concepts.
Mastering LangChain: Open-Source Models & Prompt Engineering (Part 2) was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.