\documentclass{article}      % Specifies the document class
%
%                            % The preamble begins here.
\title{Homework 1}  % Declares the document's title.
%\author{John Lambadaris}      % Declares the author's name.
\date{September 24, 2010 }      % Deleting this command produces today's date.
%\newcommand{\ip}[2]{(#1, #2)}
                             % Defines \ip{arg1}{arg2} to mean
                             % (arg1, arg2).
%\newcommand{\ip}[2]{\langle #1 | #2\rangle}
                             % This is an alternative definition of
                             % \ip that is commented out.
\begin{document}             % End of preamble and beginning of text.
\maketitle                   % Produces the title.
%\section{Ordinary Text}      % Produces section heading.  Lower-level
                             % sections are begun with similar 
                             % \subsection and \subsubsection commands.

\begin{enumerate}
\item Show that a process with independent increments has the Markov property. The reverse is not true! (Provide a counter-example)
\end{enumerate}
The following are problems related to Markov Chains
\begin{enumerate}
\item Prove the Chapman-Kolmogorov equation: \(\textbf{P}^{\textbf{(m+n)}}=\textbf{P}^\textbf{m} \cdot \textbf{P}^\textbf{n} \;\; m,n=1,2,3... \)
\item Prove the renewal equation of page 6 (in the notes)
\item Show that any state in an irreducible Markov Chain with finitely many states must be positive recurrent
\item To look backwards in time, write \(q_{ij}= P(X_n=j \mid X_{n+1}=i)\). Show that if the chain starts at time 0 in
a state according to the stationary probability distribution \({\underline \pi}\) (i.e. \(P(X_0=i)=\pi_i\)) then  \(q_{ij}= {\pi_i p_{ij} \over {\pi_i}}\)
\end{enumerate}
The following problems refer to Markov Processes
\begin{enumerate}
\item Define the Poisson process and show that is a Markov Process 
\item Show that for a Poisson process the interarrival times are i.i.d variables exponentially distributed. Furthermore, show the inverse i.e. if the interarrival times of a process are independent and exponentially distributed then the process is a Poisson process.
\item Work out and clean up the proof of the \emph{independent observer property} of the Poisson process as was presented on page 11 of the notes.
\item Show that the residence time at a state \emph{i} of a Markov process is a random variable that is exponentially distributed.
\end{enumerate}

\break

Note: Here are some instructions for installing the latex software. Do not install any other package except the ones listed below!
\begin{enumerate}
\item First run the installer setup 2.4.1705 file. It will ask you where it should intall Miktex from. Point it to the Miktex
directory included in the CD I gave you. The installation process is slow. Be warned.
\item After installing Miktex you may wish to install ghostview (gsv48w32) and ghostscript(gs850w32-gpl).
\item Install the WinTex XP package
\item Using WinTex XP load the example file in C:/texmf/tex/latex/base/sample2e
by pressing the "LATEX" box on the upper left of the screen. Pressing "PDFLATEX" 
will give you a PDF file. From this point you are on your own to explore and learn LATEX!
\end{enumerate}  
\end{document}               
% End of document.
