INFORMS Open Forum

Expand all | Collapse all

One More Time About a Synergy of AI and OR/MS

  • 1.  One More Time About a Synergy of AI and OR/MS

    Posted 10-13-2025 14:18
    Edited by Alexander Kolker 10-13-2025 20:47

    One More Time About a Synergy of AI and OR/MS

                I have recently attended a webinar organized by the NY Metro INFORMS Chapter on "Synergizing Artificial Intelligence and Operations Research: Perspectives from INFORMS Fellows on the Next Frontier." https://us06web.zoom.us/webinar/register/WN_HI7YOf3YSaGPYyuC5w6TzQ#/registration

                 The panelists aggressively promoted findings from their recent INFORMS Journal on Data Science paper

    Synergizing Artificial Intelligence and Operations Research: Perspectives from INFORMS Fellows on the Next Frontier

    This paper revisits Herbert Simon's 1987 vision of a partnership between artificial intelligence (AI) and operations research/management science (OR/MS) to enhance decision-making. 

    Two Heads Are Better than One: The Collaboration between AI and OR

    I have mixed feelings about this webinar and its conclusions.

    Therefore, I decided to test myself what value could be added by AI to OR/MS.

    I used, as a test-bed, my consulting work that I had performed some time ago. This example is instructive because it illustrates a typical request for analysis written in plain language with some unstructured numbers by a healthcare manager who is an analytic layman. This is a rather typical example of a problem in a healthcare setting that requires, first, some interpretation and making some reasonable assumptions to perform the analysis using OR/MS.

    I used three popular AI platforms (OpenAI GPT-4o Mini by OpenAI; Grok 3 Mini by xAI; and Free ChatGPT by OpenAI). I used, as input, the same consulting request text provided in the attached document.  The AI outcomes and recommendations are also presented in the attached document.

    The bottom line:

    ·       In my opinion, the presenters of the webinar failed to present a business case or a specific example that would clearly demonstrate the AI value added to OR/MS approach.

    ·       Simon's vision, " we should aspire to increase the impact of MS/OR by incorporating the AI kit of tools that can be applied to ill-structured non-quantitative decision domains, " was and still remains a big beautiful dream.

    ·       There were no convincing demonstrations of the value added or additional insights to OR/MS methodology by any of the three tested AI platforms. In fact, the AI recommendations were to use OR/MS methods, such as Excel spreadsheets or simulation modeling. Such recommendations were trivial and generic.

     

               

    This is the original consulting request:

    "Center Manager-Wockhardt Eye Hospital. Location: Aurangabad, Maharashtra, India.

    Dear Mr.Kolker, let me discuss my problem in detail.

    I work in an ophthalmology hospital. The ideal situation I wish is: we treat 120 patients a day with 5 doctors from 9 am to 1 pm, 2 doctors would see a preliminary examination of the eye and dilate the patient for 30 minutes, whereas 3 doctors would start seeing the final examination after the dilation and will dispose the patient. We promise a delivery of 2 hours to 80% of patients who come for watering & irritation, normal checkups, headaches, glasses, and redness. The rest 20% would have cataract, glaucoma, and other complex disease which is bound to take more than 2 hours of investigation. We can monitor the waiting time of more than 2 hours in each clinic. But this 'ideal' situation hardly happens because one doctor would be in operation theatre, one doctor would be on leave, or in ward rounds or in camp, so I am left with only 3 doctors in the unit from 9 am to 1 pm.

               With 3 doctors, the bulk of patients arrive at 10 am (40 patients/hour) all these patients have to go for vision examination by nurses (5 min each patient) and 3 nurses would do it.

    The challenges I have observed are:

    * arrival of complicated cases, where a thorough examination of the posterior segment of the eye is needed (could take up to 30 min/ patient)

    * patients who find it hard to place their chin in front of the slit lamp or language barrier, more inquisitive patients who keep loading the doctors with questions

    * patients who would go for nature's call or coffee or to the lab for blood tests and will arrive to see their successors in the doctor's consultation room before them.

    * trauma case that needs complete attention before shifting the patient to OT

    * doctors who are purposefully slow to make sure that they see fewer patients than their colleagues

    * missing case sheets by the nurse who has to spend all her time searching the case sheets.

    Now, how would I monitor the patient waiting time in a clinic of 120 patients with all these parameters, to make sure patients are seen as early as possible rather than shouting, and arguing, thus disturbing the entire atmosphere of a peaceful work environment?

    Can I work this situation with all the challenges in an Excel sheet or simulation software? How should I go about it? Kindly assist me."

    OR/MS perspective

     (see section 3.6 in the book "Healthcare Management Engineering in Action", Springer, 2024.

    https://link.springer.com/chapter/10.1007/978-3-031-53663-2_3 )

                This problem description should first be converted into a more specific and formal description that would make it amenable to quantitative OR/MS analysis.

    Below is my own problem summary that I made without any AI input:

    - clinic is open 4 hours per day, from 9 am to 1 pm. The daily patient volume is 120.

    - 80% of patients are regular ones (normal checkups, minor irritation, etc.). They are all first preliminary examined, then dilated, then finally examined and discharged.

    - 20% of patients are complicated (cataract, glaucoma, etc.). They are first preliminary examined, and then proceed for final examination by a doctor skipping dilation.

    - Preliminary examination takes 5 min, with the range 4-6 min.

    - Dilation takes to work about 30 min with a range of 20 - 35 min.

    - Dilated patients wait in a separate area until they are seen by the next available doctor for final examination.

    -Final examination by the doctor for regular patients takes about 20 min with a range from 15 min to 30 min.

    - Final examination by the doctor for complicated patients is in the range from 30 min (posterior segment of eye) to 120 min (2 hrs) (cataract, glaucoma) with the most likely being 1 hour (65 min).

    - Staffing for base (ideal case) model: 2 doctors for preliminary examination and dilation, and 3 doctors for final examination. Total doctors' staffing: 5. No nurses.

    - Staffing for the actual case is 3 nurses for preliminary examination and dilation and 3 doctors for final examination.

               Operations performance metrics target: 80% or more of regular patients wait less than 2 hours. Because of a lack of more specific information "the bulk of patients arriving at 10 am…" was interpreted as some patients (20) arriving from 9 to 10 am, while the remaining 100 (bulk) arrive at 10 am (actually spread from 10 am to 12:30 pm).  

               The question in the problem description document was "How would I monitor the patient waiting time in a clinic of 120 patients with all these parameters to make sure patients are seen as early as possible?"

                From the OR/MS standpoint, the question should have been interpreted as 'Given patient arrival pattern uncertainty, what staffing would be needed to meet the target operational performance?'

    OR/MS Solution

                Obviously, an Excel spreadsheet would not be sufficient for analyzing such a dynamic problem. Discrete event simulation modeling is the methodology that should be used to determine the optimal staffing of doctors that meets the operational performance criteria. The first model would be a baseline one (an ideal case) that includes two preliminary exam doctors and three final exam doctors. Again, I suggested this approach without any AI help. As an OR/MS expert, I did not need any help. Details of a discrete event simulation model and simulation results were presented in section 3.6 in the book "Healthcare Management Engineering in Action", Springer, 2024.

    https://link.springer.com/chapter/10.1007/978-3-031-53663-2_3

    The main take-away was that the number of doctors (not nurses) was the bottleneck. Adding full and part-time doctors will elevate this bottleneck. The model suggested the required number of full- and part-time doctors.

               It was possible to simulate many other scenarios using this type of simulation modeling to find out the most satisfactory staffing level and shifts for meeting the operational performance targets. The following simulation scenarios were suggested: (i) the effect of additional part-time exam doctors scheduled for different shifts with various availability; (ii) restriction of the total daily patient volume; (iii) changing the percentage of regular and complicated patients; (iv) various patient arrival patterns; (v) extension of clinic hours or allowing overtime; (vi) or combinations of the above.

    AI perspective.

    I used the above plain language description as an input into several popular open AI platforms to test what value AI can add to OR/MS approach or improve or enhance it in any way.

    The following AI platforms have been used:

                OpenAI GPT-4o Mini By OpenAI

    GPT-4o Mini is OpenAI's most advanced model in the small models category. It is multimodal (accepting text or image inputs and outputting text), and has higher intelligence than gpt-3.5-turbo but is just as fast. It is meant to be used for smaller tasks, including vision tasks.

                The AI output was:

    "Optimizing patient flow and monitoring waiting times in a busy ophthalmology clinic, particularly with the challenges you've described, requires a structured approach. Utilizing tools like Excel for tracking and managing data or simulation software for modeling can be effective strategies. Here's a comprehensive outline to help you navigate this issue:

    1. Data Collection and Monitoring Setup:

    Using Excel:

    - Patient Tracking Sheet:

    Create a log where you input patient arrival times, initial examination start, dilation times, and final exam times.

    - Use columns for patient ID, arrival time, start time for prelim exams, dilation duration, start time for final exams, and total waiting time.

    - Categorization:

    - Separate patients based on their complexity (routine vs. complicated cases) in different sheets/tabs.

    - Assign a complexity code (e.g., 1 for simple and 2 for complex) to each patient for sorting and analysis.

     - Nurse and Doctor Schedules:

    create a schedule that indicates the availability of each doctor and nurse. Make a note of any planned leaves or availability that may affect patient processing.

    - Incorporating Break Times/Disruptions:

    - Maintain a log for each doctor to note any disruptions (emergency cases, breaks, etc.) to identify patterns affecting wait times.

    2. Data Analysis:

    - Calculate Waiting Times:

    - Analyze the sheet daily or weekly to calculate the average waiting time, broken down by patient complexity.

    - Identify instances where patients exceed the 2-hour waiting window.

    - Bottleneck Analysis:

    - Identify common bottlenecks (e.g., wait time at vision examination, delays due to complications) and their causes.

    - Performance Tracking:

    - Track the performance of each doctor and each shift. Analyze if some doctors are consistently slower than their colleagues and identify reasons.

    3. Optimizing Patient Flow:

    Suggestion Batching:

    - Establish a triage system upon patient arrival:

    - *Fast-Track Routine Patients:* Identify lower complexity patients who can quickly proceed with exams.

    - *Complex Cases Protocol:* Set aside specific days or sessions for complex cases to reduce their impact during peak hours.

    Improvement Measures:

    - Patient Education:

    - Have educational materials ready for patients (pamphlets, videos) to reduce questions and prepare them for examinations.

    - Checking Mechanism:

    - Ensure nurses check all case sheets and necessary forms are ready before patients are called to see doctors.

    - Queue Management:

    - Use a digital queue management system (or even a physical digital board) to manage patient queues.

    4. Simulation Software:

    Using simulation software can help model various scenarios, predict patient flow, and identify optimal staffing for different times.

    1. Creating a Simulation Model:

    - Input your clinic's workflow, including patient arrival rates, doctor availability, exam times, etc.

    - Simulate different scenarios (varying the number of doctors, adding a patient education session, etc.) to see impacts on waiting times and patient satisfaction.

    Grok 3 Mini

    The latest fast model from xAI. A lightweight model that thinks before responding. Fast, smart, and great for logic-based tasks that do not require deep domain knowledge.

    The output was:

    "I understand your situation in the ophthalmology hospital-it sounds frustrating to manage patient flow with so many variables and disruptions, especially when you're short-staffed and dealing with unpredictable challenges like complicated cases, patient behaviors, and administrative issues. The good news is that you can absolutely monitor patient waiting times effectively, even with all these parameters, by using tools like Microsoft Excel for basic tracking and analysis, or simulation software for more advanced modeling. This will help you identify bottlenecks, optimize workflows, and maintain a peaceful environment without resorting to shouting or arguments.

    I'll break this down step by step: first, I'll explain how to approach the monitoring process, then discuss using Excel as a starting point, and finally, suggest when and how to use simulation software. My goal is to help you create a system that's practical, data-driven, and scalable.

    Step 1: Define Your Monitoring Goals and Process

    Before diving into tools, let's outline a clear plan. You want to track waiting times for 120 patients per day (from 9 am to 1 pm) while accounting for the challenges you mentioned. Here's how to structure this:

    - Key Metrics to Track:

    - Arrival Time: When the patient arrives (e.g., many at 10 am in bulk).

    - Preliminary Exam Time: Time for vision exam by nurses (5 minutes each) and dilation (30 minutes total per patient).

    - Final Exam Time: Time with doctors for simple cases (aim for under 2 hours total) vs. complex cases (over 2 hours).

    - Waiting Time: Total time from arrival to start of preliminary exam, plus time from preliminary to final exam.

    - Total Turnaround Time: From arrival to departure, including any delays.

    - Delays and Challenges: Note reasons for delays (e.g., complicated cases, patient interruptions, missing case sheets, doctor absences).

    - Bottlenecks: Track how factors like doctor availability (e.g., only 3 doctors), nurse workload, or interruptions affect flow.

    - Target Outcomes:

    - Ensure 80% of simple cases (watering & irritation, checkups, etc.) are seen within 2 hours.

    - Monitor the 20% complex cases (e.g., cataract, glaucoma) separately, as they exceed 2 hours.

    - Use data to forecast staffing needs, reduce disruptions, and improve patient satisfaction.

    - Data Collection Process:

    - Who Collects Data? Assign a dedicated staff member (e.g., a nurse or admin) to log data in real-time using a simple form or app. This could be done on paper initially and transferred to Excel, or directly into a digital tool.

    - How Often? Track data for every patient daily, then analyze it weekly or monthly to spot patterns.

    - Handle Challenges in Tracking:

    - For complicated cases (e.g., 30+ min exams), add a field to note the extra time.

    - For disruptions (e.g., patient delays for coffee or trauma cases), log them as "interruption events" with timestamps.

    - To address doctor variability (e.g., slow doctors), track individual doctor consultation times anonymously to identify trends without blame.

    - For missing case sheets, integrate a quick check-in system to ensure files are ready.

    By focusing on these, you'll create a "feedback loop": collect data, analyze it, and use insights to adjust schedules (e.g., stagger patient arrivals or allocate more time for complex cases).

    Step 2: Using Excel for Monitoring

    Excel is an excellent, accessible tool to start with-it's free (if you have Microsoft Office), easy to learn, and powerful enough to handle your scenario without needing advanced software. You can use it to log data, calculate wait times, and visualize trends. Here's how to set it up:

    A. Set Up Your Excel Spreadsheet

    Create a template like this (I'll describe it; you can replicate it in Excel):

    1. Columns for Patient Data:

    - Patient ID: A unique number for each patient (e.g., auto-generated).

    - Arrival Time: Timestamp (e.g., 10:00 AM).

    - Patient Type: Categorize as "Simple (80%)" or "Complex (20%)" based on initial assessment.

    - Preliminary Exam Start Time: When the nurse starts (e.g., after vision check and dilation).

    - Preliminary Exam End Time: Approximately 30 minutes later.

    - Final Exam Start Time: When the doctor begins.

    - Final Exam End Time: When the doctor finishes.

    - Departure Time: When the patient leaves.

    - Delay Reasons: A dropdown or text field for notes (e.g., "Complicated case - posterior exam," "Patient interruption," "Doctor in OT," "Missing case sheet").

    - Total Wait Time: Calculated field (e.g., =Final Exam Start Time - Arrival Time).

    - Total Turnaround Time: Calculated field (e.g., =Departure Time - Arrival Time).

    Example of a basic row:

    | Patient ID | Arrival Time | Patient Type | Prelim Start | Prelim End | Final Start | Final End | Departure Time | Delay Reasons | Total Wait Time | Total Turnaround |

    |------------|--------------|--------------|--------------|------------|-------------|-----------|----------------|---------------|-----------------|------------------|

    | 001 | 10:00 AM | Simple | 10:05 AM | 10:35 AM | 11:00 AM | 11:30 AM | 11:45 AM | None | 1 hour | 1 hour 45 min |

    2. Formulas for Calculations:

    - Use Excel's time functions to automate calculations:

    - Total Wait Time: =TEXT(Final Start - Arrival Time, "h:mm") (subtracts times and formats as hours:minutes).

    - Total Turnaround Time: =TEXT(Departure Time - Arrival Time, "h:mm").

    - Conditional Formatting: Highlight cells where Total Turnaround Time > 2 hours in red, so you can quickly spot issues.

    - Averages: Add a summary section at the bottom with formulas like =AVERAGE(Total Wait Time column) to calculate daily averages.

    3. Incorporate Challenges into the Sheet:

    - Variable Patient Arrival: Use a separate tab to log arrival patterns (e.g., 40 patients at 10 am). Then, use graphs to show peaks.

    - Complicated Cases: Add a column for "Exam Duration Override" where you manually enter extra time (e.g., +30 min for posterior exams).

    - Interruptions: Create a dropdown list in the Delay Reasons column (e.g., options: "Patient Delay," "Trauma Case," "Doctor Absence"). This helps quantify how often these occur.

    - Doctor and Nurse Workload: Track which doctor/nurse handled each patient and how long it took. Use pivot tables to analyze if one doctor is consistently slower.

    - Bulk Data Entry: If you have many patients, use Excel's data validation for quick entry (e.g., dropdowns for Patient Type).

    B. Daily Workflow with Excel

    - Morning Prep: Open the Excel file at 9 am. Have staff enter data as patients are processed.

    - Real-Time Monitoring: Use a shared Excel file (via OneDrive or Google Sheets for collaboration) so you can check wait times on the fly. If a patient is approaching 2 hours, alert the team.

    - End-of-Day Analysis: After 1 pm, review the data. For example:

    - Pie chart for patient types (80% simple vs. 20% complex).

    - Line graph for wait times over the day to see spikes at 10 am.

    - This will help you answer questions like: "How many patients waited over 2 hours today, and why?"

    C. Limitations and Tips for Excel

    - Pros: It's simple, cost-effective, and you can start immediately.

    - Cons: It doesn't handle randomness well (e.g., simulating "what if a doctor is absent?"), so it's better for tracking real data than predicting scenarios.

    - Tips:

    - Train your team on basic Excel use (e.g., 1-hour session).

    - Back up data daily to avoid loss.

    - If you have 120 patients, use Excel's filters and sorting to manage the volume.

    With Excel, you can monitor your current situation effectively and start making improvements, like adjusting nurse assignments based on arrival peaks.

    Step 3: When and How to Use Simulation Software

    If Excel feels insufficient (e.g., you want to simulate "what if" scenarios with your challenges), move to simulation software. This is ideal for modeling complex systems like yours, where variables (e.g., doctor absences, patient interruptions) create uncertainty.

    Why Use Simulation Software?

    - It allows you to create a virtual model of your clinic, incorporating all your challenges to predict wait times and test solutions.

    - For example, you could simulate: "If we have only 3 doctors and 40 patients arrive at 10 am, how does that affect wait times?" Or, "What if we add a buffer for complicated cases?"

    Recommended Software:

    - Free/Entry-Level Options:

    - Excel's own simulation tools (e.g., using Monte Carlo add-ins) for simple scenarios.

    - AnyLogic Personal Learning Edition (free for basic use) or Simul8 Community Edition-these are user-friendly for healthcare simulations.

    - Paid Options: Arena or ExtendSim if your hospital can invest (they're more powerful but might require training).

    How to Set It Up:

    1. Choose Software: Start with AnyLogic-it's intuitive for beginners.

    2. Model Your Process:

    - Input Parameters: Define variables based on your description:

    - Patient arrival: Bulk at 10 am (40/hour), with 80% simple and 20% complex.

    - Resources: 3 doctors, 3 nurses; each preliminary exam takes 5-30 minutes; final exam varies (e.g., 30 minutes for simple, up to 60+ for complex).

    - Challenges: Add random events, like a 10% chance of a trauma case interrupting, or a 20% chance of patient delays.

    - Simulation Logic: Create a flowchart:

    - Step 1: Patient arrives → Goes to nurse for preliminary exam.

    - Step 2: If dilated, queues for doctor.

    - Step 3: Doctor examines; add delays for interruptions.

    - Output: Simulated wait times for 120 patients.

    3. Run Scenarios:

    - Baseline: Simulate your current setup.

    - "What If?": Test changes, like hiring a temp doctor or staggering arrivals.

    4. Analyze Results: The software will output reports, e.g., average wait times, probability of meeting the 2-hour promise.

    5. Integration with Excel: Use Excel to import simulation results for further analysis.

    Getting Started with Simulation:

    - Learning Curve: If you're new, start with online tutorials (e.g., YouTube for AnyLogic). It might take a few days to build a basic model.

    - Cost and Time: If your hospital can't afford this, stick with Excel for now and scale up later.

    - When to Switch: If you notice patterns in Excel data that need deeper analysis (e.g., frequent delays from complicated cases), simulation will help.

    Final Thoughts

    You can definitely manage this with Excel as your primary tool-it's straightforward and will give you immediate insights into waiting times while accounting for your challenges. Start by creating the spreadsheet I outlined, collect data for a week, and review it to see improvements. If things get more complex, transition to simulation software for predictive modeling.

    This approach will help you foster a more efficient, less chaotic environment by focusing on data rather than reactions. If you share more details (e.g., your current workflow or specific Excel questions), I can refine this further. Good luck-I'm here to assist!"

    Free ChatGPT by OpenAI

    Access Free ChatGPT instantly through our OpenAI API-powered interface. We've made ChatGPT's powerful AI capabilities freely available to everyone, with no registration required and no usage limits.

    The output was :

    "To effectively monitor patient waiting times in your ophthalmology clinic, especially given the complexities and challenges you've outlined, a systematic approach is important. While Excel can provide a relatively simple way to track and analyze data, a simulation software tool might be better suited for modeling the dynamic and complex nature of your clinic. Here's a step-by-step guide for both approaches:

    Option 1: Using Excel

    1. Data Collection:

    - Gather data on patient arrivals, the type of cases, time spent on examinations, and outcomes. You will need to track:

    - Arrival times

    - Time taken by nurses for preliminary examination

    - Time taken by doctors for final examination

    - Types of cases (routine or complex)

    - Any delays or disruptions (e.g., nature's call, language barriers)

    2. Create a Tracking Template:

    - Design an Excel sheet with columns such as:

    - Patient ID

    - Arrival Time

    - Preliminary Examination Start Time

    - Preliminary Examination End Time

    - Doctor Consultation Start Time

    - Doctor Consultation End Time

    - Type of Case

    - Total Waiting Time

    - Notes (for delays or other disruptions)

    3. Formulas:

    - Use formulas to calculate waiting times. For instance:

    - Total Waiting Time = (Doctor Consultation End Time - Arrival Time)

    4. Data Analysis:

    - Use pivot tables or graphs to analyze waiting times based on different parameters (e.g., types of cases, doctor efficiency).

    - Highlight any trends, such as peak times leading to delays or specific issues causing longer waiting times.

    5. Regular Reviews:

    - Set a schedule to review the data regularly, and adjust workflows or resource allocation based on findings.

    Option 2: Using Simulation Software

    1. Choose a Simulation Tool:

    - Select software that specializes in healthcare operations, such as AnyLogic, Simul8, or Arena, which can model patient flow.

    2. Model Your Clinic:

    - Create a model that includes:

    - The flow of patients from arrival to consultation.

    - Time distributions for examinations (e.g., 5 minutes for preliminary, 15-30 minutes for final depending on complexity).

    - Random arrivals based on historical data.

    - Include factors for delays, such as patients taking breaks, language barriers, and others.

    3. Input Data:

    - Input your collected data (arrival rates, processing times, etc.) into the simulation model.

    4. Run Simulations:

    - Simulate various scenarios to observe how changes in staffing levels, patient arrival patterns, and case complexities impact waiting times.

    - Explore "what-if" scenarios, such as what happens when one doctor is unavailable or if more nurses are added for preliminary exams.

    5. Output Analysis:

    - Analyze the results from the simulation to identify bottlenecks and areas for improvement.

    - Use the insights gained to inform changes in scheduling, staffing, and processes.

    General Recommendations

    1. Standard Operating Procedures (SOPs):

    - Standardize processes for routine examinations and consultations to minimize variations caused by different doctors.

    2. Patient Education:

    - Provide clear instructions to patients before their appointment regarding what to expect, helping them to be prepared (e.g., time for blood tests or what to do with delays).

    3. Communication:

    - Implement a system for keeping patients informed about expected waiting times and managing their expectations.

    4. Resource Allocation:

    - Consider hiring additional staff during peak hours or cross-training existing staff to fill in as needed.

    5. Feedback Mechanism:

    - Create a system for collecting patient feedback on waiting times and experiences to continuously improve the process.

    By utilizing either Excel for basic tracking or a more advanced simulation for in-depth analysis, you can gain valuable insights into your clinic's operations, ultimately leading to improved patient satisfaction and efficient use of resources."



    ------------------------------
    Alexander Kolker
    ge healthcare
    MILWAUKEE WI
    ------------------------------



  • 2.  RE: One More Time About a Synergy of AI and OR/MS

    Posted 10-14-2025 17:03

    Alexander,

    I sincerely appreciate that you tested a few LLMs to see what kind of assistance they would have provided you in your scenario. That's a great way to learn what works well and what doesn't. The results of your three tests didn't satisfy you that it could help. Might I suggest two things: (1) experiment with different prompts that give some guidance as to the role that the AI should assume and what it should focus on, and (2) use more recent (and better) LLMs, especially ones that have reasoning built-in. Also, since you're the expert in the particular area that the client is asking about, maybe a better metric of success for the LLM is not to compare it to what you would have done but what someone with 5-10 years experience in OR/MS would have done. If you had passed this request off to a colleague, what would you have expected to get in return? 

    For example, if I prepended this to your client's request: "You are an operations research expert with experience in healthcare. You have read the book "Healthcare Management Engineering in Action" from Springer (2024). You know the way to approach a problem with a fresh perspective and be able to use the INFORMS (informs.org) approach to job task analysis (JTA) and end-to-end solution methodology. Here is a request from a potential client. Outline ways of potentially helping this client, identifying algorithms and software needed. For each, give detailed pros and cons of the approach. Then list clarifying questions that we need to ask the client." and then added the request, submitted it to OpenAI's GPT-5 with 'Thinking' turned on, it gives a different and more detailed answer. I'll post it below - you will need to determine whether or not this is better/worse/same since I'm not experienced in this particular area, but I do think this is a great start that took only a few minutes to get some areas to pursue. As you can guess, I'm of the firm belief that AI can greatly assist in many areas, including OR/MS.

    Here is the GPT-5 output (you can also test on the pro models from X, Anthropic, etc.):

    Great brief. Below is a pragmatic, OR-driven plan you can take to leadership. I'll frame it using an INFORMS-style JTA and end-to-end methodology, then outline several complementary solution tracks-with algorithms, software, and concrete pros/cons-followed by the clarifying questions we'll need to ask.


    1) Problem framing (JTA + end-to-end method)

    Job Task Analysis (JTA) view

      • Purpose: Deliver outpatient ophthalmology consults with an 80% ≤2-hour turnaround, despite variable doctor availability and case complexity.

      • Tasks & resources: Registration → vision exam (nurses) → dilation (30 min for most) → doctor exam → diagnostics (subset) → disposition; resources: 3–5 doctors, 3 nurses, rooms/equipment, paper case sheets.

      • Workflows: Two-stage clinical flow with a fixed delay (dilation), plus branching for complex cases and trauma (priority class).

      • Performance targets: 120 pts/day in 4 clinic hours (9:00–13:00); 80% ≤2h TAT, 20% allowed >2h; minimize >2h waits by clinic; maintain smooth, calm environment.

    Method overview (INFORMS/DMAIC)

      1. Define: Map the current process (SIPOC + value stream map), define KPIs (TAT, wait time per stage, LWBS/balking, %>2h).

      2. Measure: Time-stamp each handoff; capture arrival pattern (esp. 10–11 a.m. surge), service-time distributions by case type/doctor, routing probabilities, no-shows/walk-aways.

      3. Analyze: Queueing diagnostics; bottleneck identification; variability analysis; capacity vs. demand by 15-min interval.

      4. Improve/Design: Redesign appointment/triage, staffing rosters, pooling rules, priority logic, buffer sizes; add visual controls; 5S for case sheets.

      5. Control/Sustain: Real-time dashboard, escalation rules, control charts; policy for trauma surges and low-staff days; continuous simulation A/B tests for schedule tweaks.


    2) Solution tracks (algorithms + software, with pros/cons)

    A) Queueing & capacity planning (diagnose the bottlenecks)

    What: Model the system as tandem queues: Nurse vision exam → fixed 30-min dilation delay → Doctor exam; with priority classes (trauma high, complex medium, simple low), time-varying arrivals, and server calendars (doctors on leave/OT/rounds).

    Algorithms

      • Time-varying Poisson (TVP) arrivals; G/G/c approximations (e.g., square-root-staffing, Allen-Cunneen) for waits.

      • Priority queue formulas (non-preemptive for complex; preemptive for trauma if policy allows).

      • Pooling vs. dedicated servers analysis; Little's Law to sanity-check WIP and TAT.

      • Variability impact (SCV) analysis; bottleneck identification (VUT/Kingman).

    Software

      • Excel with add-ins or templates for M/M/c / G/G/c approximations.

      • R (queueing, tidyverse) or Python (NumPy/Pandas + custom calcs).

      • Visualization with Power BI / Tableau.

    Pros

      • Fast, low cost, immediate insights.

      • Good for "why is 10–11 a.m. painful?" and staffing what-ifs.

      • Validates feasibility of 120 pts/4 hours with 3 vs. 5 doctors.

    Cons

      • Approximations; hard to capture rework, wandering patients, room constraints, and case-sheet losses.

      • Doesn't show minute-by-minute congestion as realistically as simulation.


    B) Appointment and arrival smoothing (change the input)

    What: Redesign appointment templates and walk-in triage to flatten the 10 a.m. surge; pre-dilation for qualifying patients; stagger doctor start times; buffer slots for trauma/complex.

    Algorithms

      • Stochastic appointment scheduling with no-shows and dilation lag; objective: minimize mean/95th-percentile TAT.

      • Overbooking optimization by time block (chance-constrained).

      • Slot mix optimization (simple/complex ratios by 15-min block).

    Software

      • Python OR-Tools / PuLP (open-source LP/MIP), or Gurobi/CPLEX if available.

      • Export to Excel scheduling templates for daily use.

    Pros

      • Biggest lever: fewer giant peaks → shorter queues everywhere.

      • Works even when only 3 doctors are present.

    Cons

      • Requires culture change and booking discipline.

      • Needs reliable complexity triage at scheduling/arrival.


    C) Real-time flow control & digital queuing

    What: Token-based single queues per stage (avoid "choose your doctor" micro-queues), real-time status boards, SMS recalls for patients who leave for tests/coffee, fixed WIP caps at each step.

    Algorithms

      • Dispatch rules: Shortest-processing-time-first for simple pool; triage priority for trauma/complex; pull system from doctor stage to start pre-dilation only when downstream WIP is below cap.

      • Workload leveling (Heijunka) logic.

    Software

      • Lightweight kiosk/QR time-stamping + Google Apps Script/Power Apps; or off-the-shelf queue apps.

      • Real-time dashboards in Power BI/Tableau.

    Pros

      • Immediate wait-time visibility; fewer arguments at the desk.

      • Reduces "overtaking" by wanderers; controlled WIP lowers TAT variability.

    Cons

      • Needs basic IT support and staff training.

      • SMS/WhatsApp workflows must match local patient behavior/languages.


    D) Discrete-Event Simulation (DES) of the clinic

    What: Build a minute-level simulator of your clinic with actual distributions, calendars, and routing (including dilation, complex cases up to 30 min, trauma preemptions, nurse/room constraints, case-sheet mishaps).

    Algorithms

      • DES engine with entity routing, resource seize/release, calendars, priorities, balking/reneging.

      • Simulation-optimization (search over appointment templates, staffing, WIP caps).

    Software (choose one)

      • Simul8 Healthcare, AnyLogic, Arena (paid; healthcare libraries, GUIs).

      • SimPy (Python) or simmer (R) (open-source; code-based, very flexible).

      • JaamSim (free GUI) if budget-constrained.

    Pros

      • Captures all your "messy realities": wanderers, missing sheets, language delays, doctor-specific speed, trauma preemptions.

      • Lets you test policies safely before rollout; produces persuasive visuals.

    Cons

      • Needs clean input data and modeler time.

      • Governance required to keep it updated (avoid "one-off model" trap).


    E) Staff rostering and day-of reallocation

    What: Build schedules that anticipate OT/leave/camps; cross-train/float roles; day-of robust reallocation when doctors drop from 5 → 3.

    Algorithms

      • Integer programming for monthly/weekly rosters with coverage constraints and fairness.

      • Robust optimization or stochastic programming to hedge against absences.

      • Real-time rebalancing heuristic using current WIP + forecast arrivals.

    Software

      • OR-Tools CP-SAT, Gurobi/CPLEX; front-end in Excel or a small web app.

    Pros

      • Converts variability in staffing into planned buffers.

      • Fairness rules reduce "purposefully slow" behavior incentives.

    Cons

      • Requires accurate availability inputs and policy alignment.

      • Optimization outputs must be interpretable and editable by managers.


    F) Process reliability: case sheets, 5S, and language support

    What: Reduce "special-cause" delays (lost sheets, difficult positioning, language).

    Algorithms/tools

      • 5S/visual management for physical files; Kanban for sheet movement.

      • Standard work times with control charts (X-MR) on vision and exam durations.

      • Pareto analysis for delay causes; FMEA for failure modes (lost sheets, device downtime).

      • Interpreter routing (rules-based) and pictorial instruction cards for slit-lamp positioning.

    Software

      • Excel/Power BI control charts; lightweight barcode or RFID for case-sheet tracking.

      • Process mining (if EHR timestamps exist) to spot rework loops.

    Pros

      • Cheap, immediate waste removal; directly tackles stated pain points.

      • Stabilizes inputs for the DES and scheduling models.

    Cons

      • Lean changes require daily management; benefits erode without audits.


    G) Performance transparency & incentives (behavioral)

    What: Case-mix-adjusted productivity and fairness so no one "wins" by going slow.

    Algorithms

      • Multilevel regression to adjust throughput for case complexity and patient mix.

      • Risk-adjusted control charts per doctor; CUSUM to detect sustained drift.

      • Equitable load balancing rules in the dispatcher (pooled queue).

    Software

      • R/Python analytics; publish in Power BI/Tableau.

    Pros

      • Reduces gaming; focuses debate on data, not personalities.

      • Supports coaching and targeted training.

    Cons

      • Needs trust and careful rollout; ensure measures don't encourage cutting corners.


    H) Monitoring: how to track waiting time for 120 patients

    Minimal viable system

      • Time-stamps at: arrival, start/end vision, start dilation, start/end doctor, diagnostics start/end, disposition. Use QR/short codes on the case sheet or patient token; staff scan at each step (Android phones suffice).

      • Live dashboard: queue lengths per stage, % within 2h today, 95th-percentile wait, per-clinic red/yellow/green.

      • Escalation rules: if predicted >2h exceeds 20% for the next hour, trigger actions (pull in backup doctor, increase simple-case ratio, extend clinic by 15 min, etc.).

    Algorithms

      • Short-horizon forecasting of arrivals using moving averages; Erlang-C/G/G/c to predict downstream waits; alerting thresholds.

    Software

      • Google Sheets + Apps Script or Power Apps for capture; Power BI for live visuals.

      • Scales later to an EHR-integrated workflow.

    Pros

      • Fast to deploy; creates the data spine needed for all other tracks.

      • De-escalates arguments-staff can point to the board, not each other.

    Cons

      • Requires consistent scanning; start simple (arrival + doctor start) and add steps.


    3) Excel vs. simulation: when to use which

    Excel first (weeks 1–2): Queue approximations, simple staffing calculators, appointment templates, daily dashboards.
    Simulation next (weeks 3–8): Validate policies with all complexities; run "5→3 doctors" stress tests; design robust playbooks for trauma surges and staff shortages.
    Optimization overlays (weeks 4–8): Slot mix, overbooking, and roster optimization informed by the simulation.


    4) Quick-win interventions to pilot (low cost)

      1. Single pooled queue per stage with token display; ditch room-level micro-queues.

      2. Pre-dilation protocol for clearly simple follow-ups (per clinical policy) to decouple from doctor availability.

      3. Stagger starts (e.g., one doctor at 8:45 for early trickle/trauma buffer; one at 9:15) + buffer slots at 10:00 to absorb the surge.

      4. WIP caps: limit number in dilation buffer and waiting-for-doctor; start pulls only when below cap.

      5. 5S for case sheets + barcode stickers; daily audit.

      6. Interpreter cards and slit-lamp coaching visuals to reduce handling delays.

      7. Daily readiness huddle at 8:50 with forecast and contingency (who is on call if we drop to 2 doctors).


    5) Data we'll ask for (minimal to start)

      • Arrivals by 15-min over 4–6 weeks; share of walk-ins vs. booked; no-show rate.

      • Service times (min, mean, 75th/90th/95th percentiles) for: vision exam, doctor exam (simple vs. complex), diagnostics; by doctor if possible.

      • Routing fractions: % needing dilation, diagnostics, procedures; % trauma.

      • Resource calendars: doctor/nurse availability, rooms/equipment counts.

      • Current policies: triage criteria, trauma handling, "overtaking" rules.

      • Outcomes: current %≤2h, LWBS/balking, patient complaints.


    6) Clarifying questions for the client

    Demand & mix

      1. What % of patients truly require dilation, and can any be pre-dilated safely by protocol?

      2. Distribution of simple vs. complex vs. trauma across the day? Are cataract/glaucoma largely pre-booked or walk-ins?

    Times & resources
    3) Observed time distributions (not just averages) for nurse vision exam, dilation-to-doctor lag beyond 30 min, doctor exam by case type, diagnostics.
    4) Number of rooms/slit lamps and any shared equipment constraints.
    5) Typical doctor availability patterns (OT days, rounds, camps) and same-day variance.

    Policies & priorities
    6) Current rules for trauma preemption-can trauma interrupt ongoing consults or is it handled by a designated doctor?
    7) Are patients allowed to "hold" their place when they leave for labs/coffee? What is the formal policy vs. actual practice?

    Scheduling
    8) Share of booked vs. walk-in; appointment template granularity; overbooking currently used?
    9) Show-up reliability by time of day/day of week; any patterned late arrivals (e.g., 10–11 a.m. peak due to transport)?

    Information flow
    10) Do you have EHR time-stamps today? If not, is scanning/QR feasible at each station?
    11) Frequency and causes of missing case sheets; can we attach barcodes and define handoff checkpoints?

    People & incentives
    12) How do you measure doctor productivity now? Are you open to case-mix-adjusted dashboards and pooled queuing to reduce gaming?
    13) Language mix of patients; access to interpreters or bilingual staff?

    Constraints & goals
    14) Acceptable overtime? Willingness to shift some volume to 8:45–9:00 pre-clinic or 13:00–13:15 spillover?
    15) Primary success metric: 80th percentile ≤2h, 95th percentile ≤? Specific targets per clinic?


    7) Recommended stack (practical)

      • Data capture & dashboards: Google Forms/QR → Google Sheets → Power BI/Tableau.

      • Queue analysis: Excel + Python/R notebooks for G/G/c and priority models.

      • Simulation: Start with SimPy (Python) for flexibility; if budget allows, Simul8 Healthcare for faster stakeholder buy-in.

      • Optimization: OR-Tools (open source) or Gurobi if available; deployment back to Excel templates for scheduling/rosters.


    8) What success looks like (within one quarter)

      • Live wait-time board and daily huddles; staff using WIP caps and pooled queues.

      • Appointment template that flattens the 10–11 a.m. spike; buffer for trauma.

      • Case-mix-adjusted productivity view; less variability between doctors.

      • ≥80% ≤2 hours on >80% of clinic days, even when staffing drops to 3 doctors.

    If you'd like, I can draft the initial Excel capacity calculator (arrival by 15-min × G/G/c wait estimator) and a SimPy skeleton that mirrors your dilation flow and priority classes, using placeholder distributions until we get your real data.



    ------------------------------
    Warren Hearnes, PhD
    Founder, OptiML AI
    INFORMS Board Role: VP Technology Strategy
    ------------------------------



  • 3.  RE: One More Time About a Synergy of AI and OR/MS

    Posted 10-15-2025 11:13

    Warren, I appreciate your taking the time to write your detailed response and suggestions to my post regarding the role of AI for possible synergy with OR/MS.

    I think the key to our different assessment of the AI role as an enabler and value-added component to OR/MS is in your phrase: "As you can guess, I'm of the firm belief that AI can greatly assist in many areas, including OR/MS." Of course, after all, you are the Founder, OptiML AI.

    Now, as you can guess, I have an open mind about the value AI, but at this time I am highly skeptical of your belief. But the issue is not someone's personal 'belief' or skepticism. In my consulting capacity, my goal is to deliver to the customer the specific solution for the posed problem and to convince the customer that I chose the most efficient and direct methodology for doing so. With regard to AI, I asked myself: "Do I really need an assistant for solving the specific problem posed to me, be that assistant AI or a human?"  And my answer is resounding no.

          You suggested to me "two things: (1) experiment with different prompts that give some guidance as to the role that the AI should assume and what it should focus on, and (2) use more recent (and better) LLMs, especially ones that have reasoning built in." Your suggestions completely changed my and the customer's goals and the subject: instead of focusing on the solution of the customer's business problem, you suggested testing some internal AI issues, such as different prompts, the role that AI should assume, and testing better LLMs. However, I am not interested in spending more of my time exploring the role and the potential of AI with little or no chance of getting additional value. I am interested in solving the customer's business problem using the most appropriate methodology, as I judge it. This difference in focus illustrates the core of the "AI rage" issue: a lot of resources are already spent on AI (LLMs) development. AI promoters, marketers, and investors are desperate to demonstrate 'amazing' applications and the usefulness of this "big beautiful thing". This is the reason why AI is claimed to be an almost universal silver bullet that can assist and significantly enhance human ability for solving difficult problems. AI could potentially be useful in solving ill-defined problems with contradictory multiple goals, non-quantifiable or difficult to formalize factors expressed in natural language, generating new alternative hypotheses, etc. But I am not familiar with examples of such AI applications.  

                   Your GPT-5 output included in your reply is a useless word salad that includes everything, from generic JTA to Six Sigma DMAC, from queuing & capacity planning to the list of software with pros & cons (and who is supposed to choose pros & cons-another AI?). From DES to Process Reliability with Lean tools (5S, Kanban, Control charts, FMEA, etc) that are totally irrelevant in this context. And so on and on, page after page that makes an illusion of the deep expertise and value, but actually is a hollow babbling close to hallucination.

    Anyway, thank you for the opportunity for discussion.



    ------------------------------
    Alexander Kolker
    ge healthcare
    MILWAUKEE WI
    ------------------------------



  • 4.  RE: One More Time About a Synergy of AI and OR/MS

    Posted 10-16-2025 11:26

    Alexander:
    I admit that I have my own reservations about using LLMs and GenAI, and that care must be taken when using this new technology.   You wrote:


    In my consulting capacity, my goal is to deliver to the customer the specific solution for the posed problem and to convince the customer that I chose the most efficient and direct methodology for doing so.

    I think that Warren is suggesting that a GenAI/LLM can assist you in determining that you have made the right decisions in how you approached your customer's problem.  Here, the AI is not the problem solver.  But it can be a handy assistant.

    You might want to take a look at this article from OR/MS Today:  https://pubsonline.informs.org/do/10.1287/orms.20 written over a year ago about an experiment done by a workgroup that used ChatGPT to help guide a solution to a proposed analytics problem. Both this article and Warren are suggesting that leveraging this technology can help us as OR practitioners.

    As a practitioner, I am on the lookout for applications of GenAI/LLMs that truly deliver value and that justify the investment of time and money. One way is by making our own work easier, which is the thing you have experimented with, and is illustrated in the aforementioned article.  But another is how we can use the technology in the solutions that we deliver to clients so that we enhance the client experience.  One such example is described nicely in this INFORMS MSOM webinar from people at Microsoft:  https://www.informs.org/Resource-Center/Video-Library/Videos-for-Topic-Pages/M-SOM-Practice-Platform-Using-Optimization-and-LLMs-to-Enhance-Cloud-Supply-Chain-Operations .  I believe we'll see additional examples like this in the future.



    ------------------------------
    -Irv Lustig
    Optimization Principal
    Princeton Consultants
    ------------------------------



  • 5.  RE: One More Time About a Synergy of AI and OR/MS

    Posted 10-16-2025 13:35

    Irv,

    Thank you for your feedback.

    1st, the link that you suggested https://pubsonline.informs.org/do/10.1287/orms.20

    does not work, returning 'An Error has occurred, please contact your System Administrator.' With no exact reference to the authors, I am unable to read this paper.

    2nd, you write "Warren is suggesting that a GenAI/LLM can assist you in determining that you have made the right decisions in how you approached your customer's problem.  Here, the AI is not the problem solver.  But it can be a handy assistant."

    But I made clear in my post, "With regard to AI, I asked myself: "Do I really need an assistant for solving the specific problem posed to me, be that assistant AI or a human?"  And my answer is resounding no."

    Certainly, everybody (even the most experienced consultant) can make a mistake in choosing the best approach to solving the client's problem.  However, what makes you think that AI will better "help guide a solution to a proposed analytics problem. Both this article and Warren are suggesting that leveraging this technology can help us as OR practitioners."?? Why do I have to trust AI more than my own judgment and experience? How do we know, as OR practitioners, that AI technology will truly "enhance the client experience'??

    I see the whole issue from a different angle: if someone, be it AI or a human, suggests something, I will have to spend my time verifying the validity and value of this suggestion. After all, I am responsible for the outcome of my work, not AI nor anyone else. I

    Thanks again



    ------------------------------
    Alexander Kolker
    ge healthcare
    MILWAUKEE WI
    ------------------------------



  • 6.  RE: One More Time About a Synergy of AI and OR/MS

    Posted 10-16-2025 14:03

    Sorry, a copy/paste error.  The link is https://pubsonline.informs.org/do/10.1287/orms.2024.02.03/full/

    You wrote:


    "With regard to AI, I asked myself: "Do I really need an assistant for solving the specific problem posed to me, be that assistant AI or a human?"

    I think the more appropriate question is "Would ideas from an assistant for solving the specific problem posed to me be useful, be that assistant AI or a human?"  

    At least in my experience, collaborating with colleagues on solutions for specific problems is quite valuable, and an AI assistant becomes just another collaborator providing ideas that I nor my other collaborators might have thought of.



    ------------------------------
    -Irv Lustig
    Optimization Principal
    Princeton Consultants
    ------------------------------



  • 7.  RE: One More Time About a Synergy of AI and OR/MS

    Posted 10-16-2025 20:09

    Irv,

    In general, I agree with your suggestion to modify my question from "Do I really need an assistant..." 

    to "Would ideas from an assistant...be useful ...". The critical issue, however, is that not all assistants are created equal. Collaborating with your colleagues/assistants or AI can be fruitful and valuable only if your collaborators are qualified and/or possess the expertise that you lack. If, however, your collaborators are not qualified enough, they become a liability and distraction. Your performance will suffer. This is what I learned during my career when some so-called assistants were forced on me by the upper management without my input on their qualifications. I recognize that you might have a different experience. That's OK.

    I'd like to illustrate my point using Chat GPT as a collaborator/assistant using a simple OR/MS problem from my book "Healthcare Management Engineering in Action", Springer, 2024. Healthcare Management Engineering In Action: Applying Fundamental Management Principles for Operational Decision Making in Healthcare | SpringerLink

    This is a short problem description from section 2.9:

     "In many clinics, mixed patient arrival patterns frequently exist, i.e. some patients are scheduled to arrive at a specific time while other patients arrive unexpectedly at random points in time. Suppose one procedure room is available and there are eight procedures scheduled for a day, at 6 am, 8 am, 10 am, 12 pm, 2 pm, 4 pm, 6 pm, and 8 pm. On this day for 24 hours six random emergency patients also arrived with the average inter-arrival time of 4 hours. The average procedure time for both for simplicity is 1 hour. What is the average number of patients in the queue and their expected wait time?" 

    I used this short text as a prompt for free Chat GPT.

    This is the outcome: "To determine the average number of patients in the queue and their expected wait time, we can use queuing theory principles. Only the emergency patients seem to create a queue, as the scheduled procedures occur at fixed intervals and will not interfere with each other initially. Thus, we focus primarily on the random arrivals (emergency patients). The system can be modeled as an M/M/1 queue (which means it has Poisson arrivals and exponential service times with a single server). [Then some M/M/1 queuing formulas were included with the final summary]:

    Summary
    Average number of patients in the queue \(L_q \approx 10.58\) patients
    Expected wait time in the queue \(W_q = 11.5\) hours

    If it was an undergraduate OR/MS student, I would certainly grade this answer Fail (F).

    This AI assistant/student does not understand the basic OR/MS management principles concerning the application of queuing theory, namely that queuing analytic formulas should not be used if the arrival flow contains a non-random component, i.e. it is not a Poisson random arrival. But in this problem only 6 patients are the Poisson random, and 8 patients are non-random scheduled, but my AI assistant treated all as random, grossly overestimating the actual wait time and the queue size. 

    This is a reflection of the general management engineering principle: the lower the degree of randomness in arrival rate and service time the higher the unit operational performance in terms of reduction the queue size and wait time.

    It did not occur to this 'assistant' that discrete event simulation model must be used for this type of problems with mixed arrivals. I included in my book multiple examples illustrating fundamental management principles and summarized them in the final chapter. This simple example illustrates again my point: a half-educated unqualified 'assistant' is a liability, not helper. 

    Will an AI LLM model be educated enough in the future as a real OR/MS enabler is still unclear. Currently, it is a dream.



    ------------------------------
    Alexander Kolker
    ge healthcare
    MILWAUKEE WI
    ------------------------------



  • 8.  RE: One More Time About a Synergy of AI and OR/MS

    Posted 10-17-2025 21:19

    Hello Alexander,

    My experience with LLMs such as ChaptGPT, Gemini, and Copilot is that their answer is heavily dependent on the way the prompt is engineered and also on the version of the LLM (the more recent versions perform better).   I posed the prompt  "Act as an OR expert.  'your problem statement'.  Recommend a solution method" to these three LLMs.   In all cases, their proposed solution method was discrete event simulation (see below for their answers). 

    Sincerely,

    Mehran Hojati

    ChaptGPT 5

    Here's the simplest, most reliable way to handle this mixed (appointments + walk-ins) clinic:

    Recommended method: Discrete-Event Simulation (DES)

    Why: arrivals are time-dependent (scheduled blocks + random emergencies), which breaks most neat closed-form queues. A tiny DES gets exact averages under your rules (FCFS vs priority, preemption, no-shows, overtime, etc.).

    Minimal plan (FCFS, non-preemptive, 1 room, 1-hr service)

    ....

    Gemini 2.5

    I can certainly act as an Operations Research (OR) expert for this problem. This is a classic mixed-arrival queuing system-combining deterministic scheduling with stochastic emergencies-which makes standard queuing formulas inapplicable.

    The most appropriate and recommended solution method is Discrete-Event Simulation (DES).


    Recommended Solution Method: Discrete-Event Simulation (DES)

    Why DES is Necessary

    Standard queuing models (like M/M/1) assume Poisson arrivals (completely random). Your system violates this assumption because 8 of the 14 total patients are scheduled for specific times. Simulation is required because it can explicitly model the exact timing of all events, capturing the complex interaction between:

    1. Deterministic Arrivals: Patients who arrive exactly at $6 \text{ AM}, 8 \text{ AM}$, etc.

    2. Stochastic Arrivals: Emergency patients arriving randomly based on a probability distribution (likely a Poisson process, given the average inter-arrival time).

    3. Single Server: The one procedure room.

    4. First-Come, First-Served (FCFS) Queue: The implicit queuing discipline.

    How to Model the System in a Simulation

    ...

    CoPilot

    This is a rich and realistic operations research (OR) problem involving a single-server queue with mixed arrivals-deterministic (scheduled patients) and stochastic (emergency patients). Here's how an OR expert would approach it:

    🧠 Problem Summary

    • Server: One procedure room

    • Scheduled arrivals: 8 patients at fixed times (every 2 hours from 6 AM to 8 PM)

    • Emergency arrivals: 6 patients randomly over 24 hours, modeled as a Poisson process (mean inter-arrival time = 4 hours)

    • Service time: Constant, 1 hour per patient

    • Goal: Estimate average queue length and expected wait time

    ✅ Recommended Solution Methods

    1. Discrete-Event Simulation (DES)

    This is the most flexible and accurate method for mixed arrival patterns.

    • Why: Handles time-dependent arrivals, queue buildup, and service dynamics.

    • How:

      • Simulate each scheduled arrival at its fixed time.

      • Generate emergency arrivals using exponential inter-arrival times.

      • Track queue length and wait time minute-by-minute or event-by-event.

    • Tools: Python (SimPy), R (simmer), AnyLogic, Arena

    This method lets you test variations: priority rules, overtime, cancellations, etc.



    ------------------------------
    Mehran Hojati, Phd

    ------------------------------



  • 9.  RE: One More Time About a Synergy of AI and OR/MS

    Posted 10-18-2025 00:06

    Mehran, I appreciate your time testing three LLMs with the updated, improved prompt.

    The issue, however, is whether AI (LLMs models) are capable of providing added value, or new insights that escaped a qualified OR/MS specialist.

    From this standpoint, these tested LLMs with improved prompts failed to deliver anything new or valuable. Yes, they made generic recommendations for using discrete event simulation rather than queuing analytic closed formulas, like with my original prompt. However, any qualified OR/MS specialist will immediately recognize that discrete event simulation is the only valid methodology for these types of problems. No AI LLMs are needed to see that.  Thus, using AI LLMs for these and for other, more complex problems is a waste of time and a distraction for a qualified OR/MS specialist. Multiple OR/MS examples taken from healthcare practice and applications of various OR/MS methodologies are presented and illustrated in the book Healthcare Management Engineering In Action: Applying Fundamental Management Principles for Operational Decision Making in Healthcare | SpringerLink  

    The bottom line is that you actually confirmed my point: there is no evidence and realistic examples that a synergy between even the most advanced AI LLMs and OR/MS adds any value to a qualified OR/MS specialist.

    In the distant future, more advanced LLMs could be developed and add value to solving some ill-defined OR/MS problems with non-quantifiable and difficult to formalize factors, or generating some new alternative criteria for multi-objective decision-making problems, etc. But currently, it is just a beautiful dream.

    Thank you for your input on this discussion.



    ------------------------------
    Alexander Kolker
    ge healthcare
    MILWAUKEE WI
    ------------------------------



  • 10.  RE: One More Time About a Synergy of AI and OR/MS

    Posted 10-18-2025 01:39
    Edited by Rahul Saxena 10-18-2025 01:44

    Like Alexander, LLM used as analytics assistants proved to be disappointing for me because of the slop they generate, similar to the distraction of having an unqualified assistant.

    My software engineer colleagues use LLMs in the way Irv suggested, and they find it useful (paid subscriptions to Claude). They say they have to do what Irv says: take the LLMs response and think it through. But that it's worthwhile when they're scanning and exploring. 

    Why don't I find LLMs useful? Maybe because I have to hold my concept of the customer requirements in my mind as I work through what's needed to optimize and criticize. Both are edge-finder needs. I understand that LLM are centroid-finders.

    So my list of LLM uses in decision intelligence is:

    1. Software engineering assistant
    2. Report summary with "anti hallucination" prompts

    Data transformation is a concept-heavy stage where LLMs don't make any useful contribution now. But once several data-pipeline specs exist to guide the LLM we plan to see if LLMs could help to build first-pass data mapping and data-transform specs that can be checked deterministically and run through our ETL functions. 



    ------------------------------
    Rahul Saxena
    RevInsight.com
    ------------------------------



  • 11.  RE: One More Time About a Synergy of AI and OR/MS

    Posted 10-18-2025 11:53

    I'd like to thank everybody who contributed to this ongoing discussion and expressed their views on the role of AI LLMs in OR/MS: Warren Hearness, Irv Lustig, Mehran Hojati, Rahul Saxena.

    I'd like to present here an example of the problem for which, in my view, the future AI could be useful in OR/MS, in line with what I presented in my earlier comments. Of course, one may define many other really difficult OR/MS problems.

    For example, it is widely accepted that healthcare value is defined, in general, as the ratio of the quality (outcomes) and the cost, V=Quality/Cost. There are many issues with the practical use of this definition. 1st, increasing the numerator and decreasing the denominator to increase V makes the ratio unstable. If the Cost becomes less and less, then the ratio goes up regardless of the numerator. At zero cost, the value goes to infinity, which is nonsense. Of course, one may say that the cost never goes down to zero, or that either the numerator or denominator should be fixed, preventing their simultaneous move. But the cost depends on who it is applied to. For a Medicaid patient, it is zero because he/she does not pay anything out of pocket. Does that mean that for the Medicaid patient value is always infinite regardless of the outcomes? For the provider or the government, the cost is always high, hence their value for the patient is low, especially with non-quantifiable quality/outcomes. This contradiction makes the value problem ill-defined (not necessarily in a mathematical sense of this term, usually applied to inverse calculation problems). 

    2nd, the numerator Quality (Outcomes) is difficult to measure. It depends on numerous and unique factors, making it impossible to calculate the value V.

    I suggest that the future AI could be asked (prompted): Define the healthcare value uniquely in such a way that it bears the same meaning both for the provider (government) and the patient? (using various versions of this prompt)

    Next prompt: define the value for both the provider and the patient that is always scaled to some standard range, such as 0 to 1 or -1 to 1? 

    Next: Can 0 be interpreted as no value at all for the patient, and the incurred cost should be returned to the payer? Can 1 be interpreted as the highest possible outcome for the patient, given the best current state of medicine? 

    Next: Can the negative value be interpreted as harm to the patient and/or medical error, bearing the provider's liability?

    Next, suggest a way to define and quantitatively measure Quality/Outcomes

    And so on. This is just an example where, I think, AI LLMs assistance could indeed add value if it is possible at all.

    Thanks again. 



    ------------------------------
    Alexander Kolker
    ge healthcare
    MILWAUKEE WI
    ------------------------------



  • 12.  RE: One More Time About a Synergy of AI and OR/MS

    Posted 10-18-2025 15:35
    Hello Alex.   I entered your email note into ChatGPT, and its response is as follows.  Thanks.  Mehran

    Here's a short, de-jargoned reply you can post that answers Alex directly and shows a concrete way AI can help with OR-not instead of it.


    Alex-thanks for pushing on a hard but important point. "Value = Quality/Cost" is indeed unstable and often unhelpful as a ratio. A practical way OR handles this (and where AI can help) is to change the math and make the definitions explicit.

    1) Replace the ratio with a net value (no division)

    Instead of , use a net benefit form:

    Here is the "willingness to pay" per unit of outcome (a policy choice).
    Why this helps:

    • No division by small/zero costs; negative values naturally mean "not worth it."

    • You can compute it for any perspective (patient, payer, provider) by plugging in that party's costs and the outcomes that matter to them.

    2) Put it on a 0–1 scale (so the numbers are comparable)

    Take Net Value and map it to with clear anchors, e.g.:

    • 0 = "no improvement over current standard (at the chosen )"

    • 1 = "best currently achievable on our data/frontier"
      A simple min–max or a logistic transform will do; pick the anchors and publish them.

    • Negative values (before scaling) already mean harm or poor value; you can display those separately or clamp at 0 on the 0–1 scale.

    3) Make "Quality/Outcomes" measurable

    Use a small set of attributes everyone understands (examples):

    • survival/time to event

    • symptoms/function (PROMs)

    • safety/adverse events

    • access/time to treatment
      Combine them with weights elicited from each stakeholder (patient, provider, payer).
      If you prefer a single summary outcome, use QALYs or a disease-specific PROM score-same framework applies.

    4) Where LLMs help (and where they don't)

    LLMs won't decide or the weights-that's policy and stakeholder work.
    But they do speed up the parts that usually slow teams down:

    • Turn messy narratives into a value tree (attributes, units, directions).

    • Draft a measurement catalog (candidate PROMs/clinical endpoints) with pros/cons.

    • Produce a small prototype that computes Net Value by perspective and shows a sensitivity slider for .

    • Generate edge cases and checks ("what happens if cost→0?", "if outcome worsens, does value go down?") so the math can't collapse.

    5) A small, concrete pilot (to make this real)

    Pick one use case (e.g., elective procedure X):

    1. Define outcomes (e.g., complication-free days, PROM score) and costs for patient/payer.

    2. Choose a provisional from the literature or a policy threshold.

    3. Compute Net Value for both perspectives; show the 0–1 scaled numbers side-by-side.

    4. Add one sensitivity plot for and one tornado chart for the outcome weights.

    This keeps the human choices (what to value, how much) visible, while letting AI draft the scaffolding and checks quickly. That's the synergy: OR provides the structure and guarantees; AI accelerates getting from idea → transparent prototype.

    If helpful, I can share a minimal spreadsheet/Python sketch that implements the net-value calculation, the 0–1 scaling with anchors, and a few built-in "sanity tests."

    You said:
    define PROM in your answer. Also, actually provide a small concrete example that would help non-healthcare OR experts understand your answer.
    ChatGPT said:

    Absolutely-here's a clearer version with the missing definition and a concrete numeric example.

    What is a PROM?

    PROM = Patient-Reported Outcome Measure.
    It's a standardized questionnaire the patient fills out about symptoms, function, or quality of life (e.g., pain, mobility, daily activities). Examples include PROMIS, EQ-5D, Oxford Knee Score. PROMs produce a numeric score, so you can measure an improvement like "+12 points after treatment."


    Why not Value = Quality / Cost?

    The ratio blows up when cost is near zero (e.g., a Medicaid patient's out-of-pocket cost), and it's hard to compare perspectives (patient vs payer).

    Replace the ratio with Net Value

    • Outcome Gain can be ΔPROM points, QALYs, or any agreed measure.

    • λ ("willingness-to-pay per outcome unit") is a policy choice that makes outcomes and dollars commensurate.

    • Works per perspective: plug in patient costs for the patient view and payer/provider costs for the payer view. No divide-by-zero; negative values mean "not worth it."

    Put results on a 0–1 scale for communication

    Once you compute Net Value for the options under a given perspective, map them to with min-max:

    • 0 ≙ worst option among those considered (or a chosen "no benefit" anchor).

    • 1 ≙ best option on today's data.

    • Negative NV (e.g., harm) stays negative before scaling and can be flagged separately.


    Concrete example (numbers you can do on a napkin)

    Elective knee problem; three choices:

    OptionPROM improvement (Δ points)Payer costPatient cost
    Surgery (S)+20$20,000$0 (Medicaid)
    Physical therapy (P)+10$1,000$0
    No treatment (N)0$100$0

    Pick λ = $1,500 per 1 PROM point (policy choice for illustration).

    Patient perspective (out-of-pocket cost = 0)

    • S:

    • P:

    • N:

    Scaled 0–1 (min=0, max=30,000): S = 1.00, P = 0.50, N = 0.00
    Note: the ratio would be infinite here (cost=0); Net Value is finite and interpretable.

    Payer perspective (actual medical spend)

    • S:

    • P:

    • N:

    Scaled 0–1 (min = −100, max = 14,000):

    • S:

    • P:

    • N:

    Takeaways

    • Patient view: surgery looks best (bigger symptom relief, zero OOP cost).

    • Payer view: physical therapy looks best (good improvement, far cheaper).

    • No contradictions-just different perspectives made explicit.

    • If a treatment harms (say ΔPROM = −5 at $5,000), then
      ⇒ clearly bad (negative).


    Where LLMs actually help (and where they don't)

    • Help: turn messy clinical notes into a value tree (attributes, units), suggest candidate PROMs, draft a small calculator (like the table above) and edge-case checks ("what if cost→0?", "what if ΔPROM<0?").

    • Don't replace: choosing λ or weights, setting clinical standards, or making policy calls-that remains expert and stakeholder work.

    If you want, I can share a 20-line Python or a simple spreadsheet that computes the Net Value and the 0–1 scores from inputs and highlights negative-value options.







  • 13.  RE: One More Time About a Synergy of AI and OR/MS

    Posted 10-18-2025 20:00

    Mehran,

    I acknowledge that your ChatGPT output in this case is better than I expected. I underestimated LLM in this particular case.

    Nonetheless, compare it with a summary from one of my projects performed well before AI LLM. (The project was not completed because of some external reasons).

    From Alex's project output summary: "Instead of the ratio with all its shortcomings discussed above, the value can be defined as a difference V=q*HALE - c*Cost, where q and c are numerical coefficients, and HALE stands for Health Adjusted Life Expectancy. At least, at the low (zero) cost, V does not go to infinity here. If q*HALE<c*Cost, then V becomes negative indicating the loss of value (waste). There is no value loss (negative V) with the ratio value [years/$]. 
    The cost is measured in $ units. The HALE is in years (time). Thus, the coefficients q and c should be in units [1/HALE] and [1/Cost], respectively
    , so that the difference V is dimensionless. Assuming the average life expectancy (from the actuarial Tables) being Y* years, q=1/Y* while c=1/C*, where C* is the CMS recommended procedure-averaged or hospital-averaged costs in $.  Value defined as a difference should be normalized to the dimensionless range 0 to 1 using the well-known normalizing transformation: V norm [0-1] =(V – V min) /(V max – V min). To normalize V to a negative value that would characterize a value loss, the normalization transformation will be:

    V norm [-1 -1] = 2*[(V – V min) /(V max – V min)]-1, where V min is negative."

    At least, your ChatGPT output is now on par with my reasoning although it is not equivalent. But it is worth to consider. I also noticed that in your output the 0 to 1 Score = max(NV) − min(NV) NV − min(NV). It looks like this is a typo: shouldn't be S=(NV-min NV)/(max NV-min NV)?

    Once again, thank you for your time, for your interest to this topic, and for your contribution.

    Best,

    Alex



    ------------------------------
    Alexander Kolker
    ge healthcare
    MILWAUKEE WI
    ------------------------------