Javascript required
Skip to content Skip to sidebar Skip to footer
Home / Pinsky Karlin 2.3.3 Solutions

Pinsky Karlin 2.3.3 Solutions

Course 02407: Stochastic processes Fall 2021

Lecturer and instructor: Professor Bo Friis Nielsen
Instructor: Tobias Overgaard
Contact: bfn@imm.dtu.dk

Textbook: Mark A. Pinsky and Samuel Karlin An Introduction to Stochastic Modelling - can be bought at Polyteknisk Boghandel, DTU. The bookstore offers a 10% discount off the announced price.
Supplementary text on conditional expectation: Zdzisław Brzeźniak and Tomasz Zastawniak, Basic stochastic processes : a course through exercises

Lectures are held in Building 358, Room 067 Tuesdays between 8.15 to 12 (E3A). Lectures will be intertwined with Exercises and Problems.

Course plan:

  • Lecture 1 - August 31: Markov Chains: Introduction
    • Reading: Pinsky & Karlin 3.1-3.5
    • A set of slides recapitulating basic probability here, for printing here.
    • Slide show (draft) in pdf, printable slides (draft) in pdf
    • Week 1.
    • Exercises and problems from Pinsky & Karlin .
    • In class exercises: KP 3.1.1, KP 3.2.1, KP 3.4.1, KP 3.4.2, (KP 3.1.4), (KP 3.2.4), (KP 3.4.6). KP exercises 3.1 , KP problems 3.1 , KP exercises and problems 3.2 and 3.4 ,
    • In class problems: KP 3.1.4 , KP 3.2.1 , KP 3.4.4 , (KP 3.2.5) , (KP 3.4.7) . Zip file with hand written solutions for exercises and problems.
    • Problems for self study: KP 3.1.1 , KP 3.2.3 , KP 3.4.12 , (KP 3.5.2) .
  • Lecture 2 - September 7: Markov Chains: Random walk, First Step Analysis, Random sums, Branching Processes
    • Reading: Pinsky & Karlin 3.6 up to 3.6.1 you only need to understand the setup and the results, 3.7, 2.3, 3.9.1-3.9.2, 3.8 can be read superficially. You can omit: 3.6.1,3.6.2,3.6.3, 3.8.1, 3.9.3
    • Slide show (draft) in pdf, printable slides (draft) in pdf
    • Week 2.
    • In class exercises: KP 3.6.1, KP 3.7.1, KP 2.3.3, KP 3.9.2.
    • In class problems: KP 3.6.7 , KP 3.8.3 , (KP 3.9.4) , (KP 3.7.1) . Zip file with hand written solutions for exercises and problems.
    • Problems for self study: KP 3.6.2 ([derive equations 3.65 only and discuss how you think the non-zero value of r will effect the mean time to absorption]), KP 3.7.4 , KP 3.8.2 , KP 3.9.7 .
  • Lecture 3 - September 14: Markov Chains: Limiting and invariant distributions, classification of states and chains
    • Reading: Pinsky & Karlin 4.1,(4.2),4.3,4.4,(4.5)
    • Slide show (draft) in pdf, printable slides (draft) in pdf
    • Week 3.
    • In class exercises: KP 4.1.1, KP 4.3.1, KP 4.4.2, (KP 4.3.2).
    • In class problems: KP 4.1.1 , KP 4.3.3 , (KP 4.1.13) , (KP 4.4.7 - to be simplified) . Zip file with hand written solutions for exercises and problems.
    • Problems for self study: KP 4.1.12 , KP 4.3.2 , KP 4.4.3 .
  • Lecture 4 - September 21: Poisson Processes
    • Reading: Pinsky & Karlin Chapter 5 - for details see weekly note
    • Slide show (draft) in pdf, printable slides (draft) in pdf
    • Week 4 .
    • In class exercises: KP 5.1.1, KP 5.3.2, KP 5.4.1, KP 5.4.3
    • In class problems: KP 5.3.7 , KP 5.1.9 , KP 5.2.6 , (KP 5.4.8) , (KP 5.3.9) , (KP 5.6.3) . Zip file with hand written solutions for exercises and problems.
    • Problems for self study: KP 5.1.7 , KP 5.4.7 , KP 5.5.6 .
  • Lecture 5 - September 28: Birth and Death Processes
    • Reading: Pinsky & Karlin 6.1-6.4 The last example of 6.4 can be omitted.
    • Slide show (draft) in pdf, printable slides (draft) in pdf
    • Week 5 .
    • In class exercises: KP 6.1.2, KP 6.2.2, 6.3.1, KP6.4.4.
    • In class problems: (KP 6.1.3), KP 6.1.4, KP 6.2.1, KP 6.2.3, (KP 6.2.6), KP 6.3.1, (KP 6.3.2), KP 6.3.3, KP 6.4.2. Zip file with hand written solutions for exercises and problems.
    • Problems for self study: KP 6.1.2 , KP 6.2.4 , (KP 6.3.4) , KP 6.4.4 .
  • Lecture 6 - October 5: Finite State Continuous Time Markov Chains, Queueing systems; basic and intermediate queueing theory
    • Reading: Pinsky & Karlin 6.6, 9.1-9.2,(9.3).
    • Slide show (draft) in pdf, printable slides (draft) in pdf.
    • Week 6. Teaching notes.
    • In class exercises: KP 6.6.2.
    • In class problems: Exercise 27, (solution), KP 9.2.2 , KP 9.2.6 , KP 9.2.4 , KP 9.3.2 . Zip file with hand written solutions for exercises and problems.
    • Problems for self study: Exercise 30, (solution) KP 9.2.5 , KP 9.2.7 , (KP 9.3.1) .
  • Lecture 7 - October 12: Pinsky & Karlin 6.5, Phase-type distributions
    • Reading: note
    • Additional Reading: Bladt & Nielsen Section 1.2.6 (discrete case) and Chapter 3,
      • (Note on Markovian arrival processes).
    • Slide show(DRAFT!) in pdf, printable slides(DRAFT!) in pdf
    • Week 7 (draft still to be finalised). Teaching notes.
    • In class problems: KP 6.5.2 , Exercise 1 (solution), Exercise 13 Q.1-Q.3 (solution), Exercise 14, (solution). Zip file with hand written solutions for exercises and problems.
    • Problems for self study: Exercise 2 (solution), Exercise 19. (solution).
  • Lecture 8 - October 26: Renewal Phenomena
    • Reading: Pinsky & Karlin 7.1,(7.2),7.3-7.5
    • Slide show (draft) in pdf, printable slides (draft) in pdf
    • Short note on moment generating function and Laplace transform .
    • Week 8. Teaching notes.
    • In class exercises: KP 7.2.1, KP 7.4.1.
    • In class problems: KP 7.1.3 , 2.9 from Wolff (solution), 2.19 from Wolff, (solution), 2.33 from Wolff. (solution). Exercise 13 Q.4 (solution), 5.22 from Bladt & Nielsen. Zip file with hand written solutions for exercises and problems.
    • Problems for self study: KP 7.3.5 , KP 7.5.4 .
  • Lecture 9 - November 2: Markov Decision Processes
    • Reading: PROBLEM WITH LINK SEE MESSAGE AT LEARN/INSIDE Martin L. Puterman: Markov Decision Processes: 1,1, (1.6), - 2.1 This section is notationally heavy, but we will only focus on the simplest type of model, which has; - in 2.1.1: Finite horizon, - in 2.1.2: finite state and action sets, - in 2.1.4 and 2.1.5: Markovian and deterministic (MD) decision rules and policies - 2.1.6 can be read superficially, but note the connection to Markov chains on the bottom of page 24. - 2.2 (except page 27), 4.1.2, 4.5 (Page 92), 4.6.4.
    • printable slides (draft) in pdf.
    • A precorona student relevant example of the secretary problem (youtube video).
    • In class problems: Puterman 2.5 (only part a) , Puterman 3.9 - Optimal parking, Puterman 3.26 - The lazy adaptable lion. Zip file with hand written solutions for exercises and problems.
    • Problems for self study: Puterman 3.10 & 3.11 Variants of secretary problem
  • Lecture 10 - November 9: Brownian Motion
    • Reading: Pinsky & Karlin 8.1-8.2
    • Slide show (draft) in pdf, printable slides (draft) in pdf
      • Small movie with Brownian motion
    • Week 10. Teaching notes.
    • In class exercises: KP 8.1.1, KP 8.2.1, KP 8.1.4, (KP 8.1.6), ((KP 8.1.5)).
    • In class problems: KP 8.1.1 , KP 8.1.2 , KP 8.1.7 , KP 8.2.1 , KP 8.2.4 , Zip file with hand written solutions for exercises and problems.
    • Problems for self study: Surplus exercises and problems from "In class".
  • Lecture 11 - November 16: Brownian Motion with Drift and Variations and Extensions
    • Reading: Pinsky & Karlin 8.3-8.4 (to Page 426)
    • Slide show (draft) in pdf, printable slides (draft) in pdf
    • Week 11. Teaching notes.
    • In class exercises:
    • In class problems: KP 8.3.3 , (KP 8.3.6) , KP 8.4.1 , KP 8.4.2 , (KP 8.4.5) , ( KP 8.4.6 ). Zip file with hand written solutions for exercises and problems.
    • Problems for self study: Brownian motion . KP 8.3.1 , KP 8.4.4 , KP 8.4.10 .
  • Lecture 12 - November 23: Black-Scholes Option Pricing Formula and Ohrnstein-Uhlenbeck Process
    • Reading: Pinsky & Karlin 8.4 from P.426 (Black Scholes), 8.5 (we might not make it to 8.5.4)
    • Short note on stationary Ornstein-Uhlenbeck process .
      • Week 12 (draft only). Teaching notes.
    • In class exercises: KP 8.4.5 , KP 8.5.1 , KP 8.5.2 , KP 8.5.3 ,
    • In class problems: KP 8.4.7 , KP 8.4.8 , KP 8.4.10 , KP 8.5.2 , KP 8.5.3 . Zip file with hand written solutions for exercises and problems.
    • Problems for self study: KP 8.5.1
  • Lecture 13 - December 1: General course overview

Deadline for reports on the take home exam is Thursday December 23nd 2021 3 pm. The exam will be available December 15

    The report has to made individually or in groups of two. It is not allowed to discuss the exercise with other students or third parties. Supplementary questions can be posted on campusnet in the relevant discussion board(s). If you hand in as a group there will be a short supplementary oral examination in January. If it is impossible for you to be physically present at DTU, then the oral exam can be held online. The report has to be handed in electronically. If you submit a scanned version of a handwritten report I will need the original not later than January 10th. The original can be sent using ordinary mail.

    Last change: 23/11 2021, by bfn

Pinsky Karlin 2.3.3 Solutions

Source: http://www2.imm.dtu.dk/courses/02407/