A MARKOV CHAIN STATE TRANSITION APPROACH TO ESTABLISHING CRITICAL PHASES FOR AUV RELIABILITY

Transcription

1 1 A MARKOV CHAIN STATE TRANSITION APPROACH TO ESTABLISHING CRITICAL PHASES FOR AUV RELIABILITY Mario P. Brito, and Gwyn Griffiths Abstract The deloyment of comlex autonomous underwater latforms for marine science comrises a series of sequential stes. Each ste is critical to the success of the mission. In this aer we resent a state transition aroach, in the form of a Markov chain, which models the sequence of stes from re-launch to oeration to recovery. The aim is to identify the states and state transitions that resent higher risk to the vehicle and hence to the mission, based on evidence and judgment. Develoing a Markov chain consists of two searate tasks. The first defines the structure that encodes the sequence of events. The second task assigns robabilities to each ossible transition. Our model comrises eleven discrete states, and includes distance-deendent underway survival statistics. The integration of the Markov model with underway survival statistics allows us to quantify the likelihood of success during each state and transition and consequently the likelihood of achieving the desired mission goals. To illustrate this generic rocess, the fault history of the Autosub3 autonomous underwater vehicle rovides the information for different hases of oeration. The method roosed here adds more detail to revious analyses; faults are discriminated according to the hase of the mission in which they took lace. Index Terms Markov rocesses, Risk Analysis, Underwater vehicles.

2 2 N I. INTRODUCTION OWADAYS the execution of many marine science rogrammes involves the deloyment and recovery of sohisticated mechatronics systems such as autonomous underwater vehicles (AUVs), remotely oerated vehicles (ROVs), or undersea gliders. Common to all of these systems is that the deloyment sequence from re-launch checks to oeration in the water to recovery consists of a series of sequential hases. With each hase there is an associated risk of loss. This risk is most obvious when the vehicle is subsurface executing its task, but is non-zero during all of the hases. For examle, while ready on deck there is a ossibility of loss due to an electrical fire. It is widely acknowledged that significant risks attend launch, and esecially recovery. Indeed some insurers require the owner to coinsure during these articular hases [1]. Although each hase involves risk, there is also the ability to reair the system after a fault, failure or incident from all but two of the hases considered. While the risks with the deloyments of these sohisticated vehicles are becoming better understood and several studies have attemted to quantify the overall risks [2]-[7], until now no framework has been roosed to structure a risk analysis for underwater vehicles by the mission hases. We roose in this aer a Markov chain aroach to model the risk in the different hases and to quantify risk for different scenarios. Markov chains were chosen because analytical results associated with the model facilitate analysis of the oerating sequences before they are generated, indicating how the oeration is likely to unfold. Furthermore, the Markov aroach allows for the essential statistical deendence between hases. The Markov model was imlemented in Matlab version 12 [8]. An early use of Markov chain theory in engineering was to estimate the reliability of hardware systems carrying out a simle task in a fixed environment [9]. The aroach was extended to model multi-hased systems [1]. The term multi-hased system is used to describe a system that either changes structure, or whose failure characteristics change during its oeration or mission. While the Markov chain resented in this aer models a structure (vehicle) that does not change, we argue that through the transitions between hases of the deloyment rocess, the failure characteristics do alter. For examle, the set of hazards, which determine risk, during launch and recovery hases are very

3 3 different to the set of hazards when the vehicle is underway. Markov chains have also been alied to the estimation of the reliability of software-based systems [11], where a model was used to generate statistical testing sequences that would ultimately hel define the reliability of a software artefact. Recent research has focused on the integration of Markov chains with other mathematical aroaches, such as Bayesian theory, Monte Carlo methods and event trees [12], [13]. The end result in all of these cases is a system that rovides an insight into the secific roblem that would be imossible to obtain by the use of Markov chains alone. Here we extend the formalism by embedding distance-related survival statistics into a Markov chain model alied to an underwater vehicle. Throughout this aer the word state is used to reference a state of the Markov model, and the word hase is used to reference a hase of the deloyment rocess, there is a one to one corresondence between a state and a hase. The aroach to devising the framework, and the questions that can be asked using the framework and Markov chain, are generic. They can be used with any item of aaratus. Here, the motivation was for the analysis of AUV mission risks, hence we draw extensively on data concerning the reliability of the Autosub3 AUV [7]. The Markov chain reresenting AUV deloyment sequence requires estimates for the transition robabilities between states. Griffiths and Trembranis [6] described an elicitation exercise for Autosub3, in which exerts were asked to assign a robability of loss for the vehicle for each of 63 faults or incidents. It was ossible to assign articular state transitions to many of the elicited robabilities. However, the judgments obtained in the elicitation exercise were not sufficient to oulate all transition robabilities of the roosed Markov chain. For examle, the study did not fully cover the re-launch tests or incidents, neither did it consider whether Loss was final, or whether salvage was ossible and, if so, whether the salvaged vehicle could be returned to service or had to be scraed. Therefore, where ossible, we elicited judgments from two colleagues, senior engineers (Stehen McPhail and Peter Stevenson) with a vast amount of exerience in Autosub3 develoment and

4 4 deloyment, for the transition robabilities which were not obtainable from the wider exert judgment study. This aer is organised as follows. Section 2 gives a brief background on the Markov chain formalism; more detail concerning this formalism is given elsewhere in the literature [14]. Section 3 resents the Markov chain model that catures the sequence of activities undertaken during the deloyment of an AUV together with details of the transition robabilities. In section 4 we resent a test case illustrating alications of the aroach and the nature of the analysis that can be suorted by the Markov chain formalism. This includes estimating robability of loss at different hases and including answers to questions an oerator might ask of the model. There is, for examle, the obvious question, Given that the vehicle assed its re-launch check what is the likelihood of a successful oen water mission of xkm?. There are also many questions related to articular hases, for instance, on the likely availability of the vehicle: Prior to re-launch tests, what is the likelihood of being able to have a mission start when needed?. Or on the likelihood of a forced early recovery: Prior to re-launch tests, what is the likelihood of having to recover the vehicle immediately after launch?. In section 5 we resent our conclusions. II. MARKOV CHAIN FORMALISM In classical robability theory a set of ossible outcomes E 1, E 2, E k. is given, with which there is associated a robability j, Pr (E j )= j ; the joint robability is defined by the multilicative roerty Pr (E 1, E 2, E k )= k. The Markov chain theory introduces an assumtion that simlifies this exression; it considers that the outcome of any trial deends on the outcome of the receding trial 1 and only on it [14]. Thus, instead of associating a robability to an event, it associates a robability to a air of events. For every air of events (E j, E k ) there is a corresonding transition robability jk, where jk is the robability of E k occurring given that E j occurred in the revious trial. According to Markov chain theory, in addition to jk, one must also define the robability of E j occurring at the initial trial, aj. Therefore for the initial trial the P{(E j, E k )}= a j jk. For the general case, considering a sequence

5 5 of many transitions, given that event E j recedes E j1 which recedes E j2 and so on for the remaining events, the joint robability distribution is comuted using the exression in 1: E j, E j1,..., E jn a j j j j j... jn j n1 jn jn P r 1 1. The exression in 1 is a result of a sequential alication of the condition robability assumtion entailed by the Markov condition. Generally seaking, a sequence of trials with ossible outcomes E 1, E 2, E k is called a Markov chain if the robabilities of samle sequences are defined by 1. Quite often a state has more than one receding state. When this is the case transition robabilities are arranged in a matrix, also denoted in the literature as a transition matrix or stochastic matrix [13]. The transition robability together with the initial state vector comletely defines the Markov chain. The matrix in 2 is an examle of a transition matrix for a Markov chain with k states. P... k k k 1k. kk 2. Considering the stochastic transition matrix in 2, 1 is the robability of moving from state to state 1, i is the robability of going from state to state i, ii is the robability of the system not leaving state i in the next ste. The stochastic matrix is useful to study the robability of a sequence of stes taking lace before the sequence is generated, and is the basis of the analyses in section 4. III. MARKOV CHAIN STATE MODEL FOR AUV OPERATIONS The deloyment rocess is reresented as a sequence of discrete states. The time sent in each state is not critical to the analysis, excet when underway, which we deal with in a different way. When underway, time is roortional to distance, and distance is an inut to a model of survival statistics. With resect to other states, it is assumed that the rocess will take as much time as necessary in order 1 Trial is here defined as an exeriment where the final outcome deends only on chance.

6 6 to decide whether or not the rocess transits to the next hase. However, when underway the transition to recovery or loss deends on the mission distance, which is generally directly related to time [1]. Other alications of Markov models have used the concet of sojourn time to model the time sent in each state [15]. At this stage we do not have sufficient information to model the time take for an AUV deloyment. It is assumed that the oerators take as much time as required in each state. A. The Markov Chain Model The Markov chain model of an AUV deloyment rocess comrises eleven states (X1 to X11), corresonding to eleven ossible hases of the deloyment. The roosed Markov chain is deicted in Figure 1; the name of each node is abbreviated to facilitate the model descrition and subsequent analysis. Where faults or incidents can lead to loss we rovide real examles, most of which have not been documented in the eer reviewed literature. The deloyment rocess starts at the initial re-test state at a deloyment location (D). This is the vehicle embarked onto a shi, or delivered to a launch site. The vehicle is ready to be tested and the ower is on. A test sequence is carried out, which comrises a set of visual and systems checks. If the tests are unsuccessful, the transition is a loo back to this state, imlying fault identification and rectification before another test is carried out in an attemt to transition to the next state. While some may argue that there are rior states that should be considered, for examle rearation at base, or selection of the oerations team, we have assumed that the effects of these factors can be, and have been, subsumed into the statistics of the hases listed. This is a simlification, and in art, a necessary simlification to enable use of the Markov aroach. In reality, selection of the oerations team can have an imact during all subsequent hases, which cannot be modelled using the Markov aroach alone. In this case a Bayesian mathematical formulation could be used for udating the transition robabilities in light of new observations concerning the exerience of the deloyment team. The next state is the ost-test state, ready to launch (Dr). The successful act of taking the vehicle from its carrier frame and into the water, be it over the side of a shi or lowered from a quay, for examle, takes the vehicle to the Overboard state (O). This may be by urose-built or general-urose gantry or

7 7 by hand for small vehicles. Problems can occur in this state necessitating recovery or they can lead to damage, for examle, due to a fire or exlosion in the energy source; although this should be a rare occurrence. The Markov chain model recognizes this ossibility with the transition from state Dr to a state where the vehicle would have to be salvaged (S). From Overboard (O) the vehicle is ready for redive checks. The vehicle is now floating in the water, as it is common for AUVs to be ositively buoyant. If the AUV asses all re-dive checks on the surface, a command is sent to the AUV to start diving (Dv). If these checks are not assed, the decision may be made to recover the vehicle (R). Oerations on the surface next to the deloyment latform carry significant risk [1]. There is the ossibility that the AUV will be lost, for examle, if the AUV is caught by the vessel s roeller, as has haened to a Norwegian AUV. During the Diving (Dv) hase some mechanical and communication tests may be carried out. New risks emerge, such as from failure of a comonent that has not been called uon until this hase, for examle a stern lane actuator whose failure can force the AUV to erform an uncontrolled dive, a situation that has lead to temorary loss (L), as haened with Autosub2 AUV in the Celtic Sea, off the UK coast. The Markov chain model catures this henomenon with the link Dv -> L. During the dive (Dv) to the holding/test attern hase (Sh) the AUV is usually within telemetry range, giving the engineers the oortunity, if necessary, to send the abort to surface signal. P 79 P 77 F X P 11 Sc X 11 P 111 S X 1 P L X 7 P 47 P 57 P 67 P 87 P 21 D X 1 12 Dr X 2 23 O X 3 34 P Dv X 4 Sh X 5 U X 6 21 P 48 P P 81 R X 8 P 68

8 8 Fig. 1. Markov state sace model caturing the sequence of events undertaken during AUV deloyment and oeration. A directional arrow from state i to state j means that the rocess can move from state i to state j. The abort to surface signal causes the vehicle to release the abort weight; this increases the buoyancy of the vehicle, forcing it to come to surface, where recovery is ossible. The submerged, holding/test attern (Sh) hase recedes the start of the mission roer. Once the vehicle has comleted the first dive, the ractice with Autosub3 is to set the vehicle in a holding or test attern, while submerged, near the deloyment latform while the telemetry is assessed. If faults are indicated, the vehicle can be recovered, but with a risk of loss as discussed earlier. All well, the vehicle roceeds with its mission, transitioning to an underway state (U). The subsequent states are either recovery (R) or loss (L). Based on mission exeriences with AUVs by the community, the lost (L) state is either ermanent, for examle with Autosub2 under an ice shelf [19], or there is a robability that a lost vehicle may be salvaged (S), through a deliberate act, e.g. using an ROV [22], or the vehicle may be found (F), through serendiity. The latter may take lace many months after the loss, as was the case in 27 with a UK Royal Navy Remus1 found after 1 months at sea. The ermanently lost state is shown by the link from L to itself. The recovery (R) may not roceed to lan as has been discussed, hence a link to Loss (L). Following either Salvage (S) or Found (F) the decision may be to scra the vehicle as being beyond economic reair (Sc), or reair and then reuse, as haened with BAE Systems Merlin. The latter is catured with the links to the initial state D. The scraed hase (Sc) is terminal. For each transition there is an associated sequence of events or conditions that must be met. A list of the conditions associated with each transition is resented in Table I.

9 9 TABLE I TRANSITION PROBABILITIES NOTATION AND CONDITIONS FOR THE MARKOV MODEL PRESENTED IN FIGURE 1. Notation Conditions for Transition Pre-deloyment check assed Pre-deloyment check not assed Deloyment over-side OK Salvaged after assing deloyment check Pre-mission diving mode entered Lost when over-side Recovery from over-side Pre-mission Holding Pattern entered Lost from Diving mode Recovery from Diving mode Underway mission started Lost from Holding Pattern Recovery from Holding Pattern Lost when underway on mission Recovery from underway on mission Recovery to deck OK Lost on recovery to deck Vehicle remains lost Vehicle found Vehicle salvaged Found vehicle reaired and reused Salvaged vehicle scraed Found vehicle scraed Salvaged vehicle reaired and reused NOTE: The subscrit ij signifies the succinct transition from state i to state j. The robability of the rocess going through a sequence of states is comuted by maniulation of the transition matrix. The transition matrix that corresonds to the Markov chain resented in Figure 1 is resented in 3. Only those elements with robabilities set out in Table 1 are non-zero. The 1 for element 1111 signifies a terminal state. P

10 1 B. Incororating transition robabilities Having set out a toology for the Markov chain that reresents the life cycle of AUV deloyments, the next stage is to determine the transition robabilities. These could be established through hard evidence through the frequentist aroach of logging the frequency of occurrence. However, in ractice, this will rarely be the case and indeed may not necessarily be the best aroach. Hard evidence is obtained easily for transition robabilities before the vehicle enters the water, such as 11, and 12. These are very likely to be robabilities determined overwhelmingly by the vehicle systems alone. In contrast, once overboard the transition robabilities are influenced by a large number of other factors. For examle, the seamanshi of the vessel s crew, exerience of the deloyment team, weather and sea conditions, the oerational environment, whether it is coastal, oen ocean, under sea ice, or under ice shelf. Hence, there is a fundamental choice of whether to oulate the matrix with hard evidence robabilities alicable only to the set of circumstances aertaining to each event (fault or incident) or whether to generalise beyond the secific case. Such a generalisation can be achieved through the technique of eliciting exert judgment [23] for the robabilities. It is the aroach taken here. In a revious risk assessment exercise Brito et al. [16] develoed an Autosub3 risk model based on its failure and incident history; a total of 63 faults were considered by a anel of exerts. Ten successful missions were also used in their analysis. This section exlains how a art of this data was used to oulate the transition matrix 3. First, it was imerative to identify, where ossible, which failures from this dataset were associated with which transition robabilities. We used the failure descrition to identify the hase of the mission in which the failure took lace. The detailed result of our assessment is resented in Aendix A, and in aggregated form in Table II, where they are indicated as EJ (Grou). The transition robability 68 is calculated based on the extended Kalan Meier statistical estimator as a function of mission distance as roosed by Griffiths et al. [18]. This is in order to model missions of different lengths, and rovides a more realistic reresentation of risk than a fixed robability. This formulation for the survival function with range r is given in 4:

11 11 1 Sˆ( r) 1 P( ei ) r r n i i where P(e i ) is the robability judgment of loss from the exerts and n i. is the failure index number. The P(e i ) variable relaces the censor flag used by the original Kalan Meier formulation [21]. The extended Kalan Meier formulation was used for calculating the transition robability 68 and through the Markov condition 67. Table II resents that survival data for oen water environment The calculated survival distribution deicted in Figure 2 was obtained using the dataset of Table II. 4. TABLE II PROBABILITIES OF LOSS USED FOR CALCULATING TRANSITION PROBABILITY P68 USING 4. Index Code Distance (km) P(e i ) Index Code Distance (km) P(e i ) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ NOTE: The failure index number is given in columns 1 and failures and 1 successful missions are considered from when the vehicle was underway. The robability judgment corresonds to the linear ool aggregated judgment, EJ(Grou). The robability judgments are for oen water environment. Ten missions that were successful, with no failures, are also included in the calculation, P(e i ) for these missions is. The Code exresses mission number and the index number of the fault on that mission, if a fault (see Aendix 1). The distribution shows for examle that for a distance of 8km the robability of survival is.97 therefore, 67 =.3 and 68 =.97. The shae of the survival distribution is similar to that obtained in revious analysis of Autosub3 failure history [16], which derived a statistical survival model for four

12 12 oerating scenarios: oen water, coastal, sea ice and ice shelf. Two risk models were built for each environment, denoted as the otimistic and essimistic. Considering the oen water scenario, for a mission distance of 8km, the essimistic model gave a robability of survival of.962, the otimistic model [16] gave a robability of survival of.992. The transition robability from the reduced dataset used here is between the otimistic and essimistic estimates of revious analyses Probability of survival Distance [km] Fig. 2. Kalan-Meier survival distribution. Source data is resent in Table III. C. Incororating exert judgment Some of the remaining transition robabilities were obtained by an exert judgment elicitation exercise that was conducted in rivate meetings with two senior Autosub3 engineers, each with more than 15 years AUV exerience, and rior exerience in assigning robabilities to suort accident investigations [19], [2]. Their judgments were aggregated using the linear oinion ool. An equal weight was assigned to both exerts [23], the aggregated robabilities are shown in Table III, indicated as EJ(McP-S). The grou of exerts (B-G) is formed by the authors of this aer. The first author has aroximately 1.5 years exerience in AUV risk analysis and the second author has aroximately 22 years exerience in AUV design, deloyment and risk assessment.

13 13 TABLE III TRANSITION PROBABILITIES FOR THE STATE SPACE MODEL Fro Transition m Stimuli state D Dr O Dv Sh U L To Stat e Source data and related information Transition Prob. P 11 D Markov condition 1-12 P 12 Dr EJ(McP-S).875 P 21 D EJ(McP-S).55 P 23 O EJ(McP-S).94 P 21 S EJ(McP-S).5 P 33 O EJ(McP-S).195 P 34 D v EJ(McP-S).97 P 37 L EJ(McP-S).15 P 38 R EJ(McP-S).9 P 45 Sh EJ(McP-S).9565 P 47 L EJ(McP-S).85 P 48 R EJ(McP-S).35 P 56 U EJ(McP-S).98 P 57 L EJ(McP-S).1 P 58 R Markov condition.19 P 67 L Markov condition 1-68 P 68 R EJ(Grou) survival P 77 L Markov condition 1-( ) P 79 F EJ(B-G).33 P 71 S EJ(B-G).33 R P 87 L EJ(McP-S).2 P 81 D EJ(McP-S).998 F P 911 Sc EJ(B-G).25 P 91 D EJ(B-G).75 S P 11 D EJ(B-G).7 P 111 Sc EJ(B-G).3 Sc P 1111 Sc Absorbing state 1 NOTE: The Markov roerty states that the sum of the robabilities leaving any given state must equal to unity. If the sum of all transitions leaving a state is constant c, where c is lower than one, than the robability of the rocess remaining in the same state in the next transition is 1-c. IV. TEST CASE OPEN WATER MISSION In this section we show how analysis of the Markov chain can address questions relevant to the risk quantification of the entire deloyment or arts of a deloyment. The results are used to identify ossible risk mitigation activities. As a test case consider an 8km long mission in oen water, with no risk mitigation activity carried out at the start of the mission, via the use of a monitoring distance.

14 14 A. Availability Availability is a measure of what fraction of those occasions on which a system is needed it is in lace and working. In the Markov toology set out here, availability is the robability of the oeration running directly from state D to state U, P(D Dr O Dv Sh U) = , is.75. As a result, the robability of not comleting this sequence is.25. This figure is affected by a number of factors, such as the vehicle ayload and the amount of testing rior to the start of the mission. Our exerts took such factors in consideration when they assigned the robability transitions. In normal conditions the science manager would exect to have a high confidence in the success of this sequence of events. However, at resent there is no rocess with the Autosub3 AUV to establish whether or not the robability of comleting the sequence is sufficiently high to meet users exectations. In contrast, the US military Unmanned Air Vehicle community has set out availability targets for a range of vehicles, furthermore actual availability data is available [24]. These, together with the result for Autosub3 derived here are shown in Table IV. TABLE IV COMPARISON OF AUTOSUB3 AVAILABILITY WITH THAT OF THREE US MILITARY UAV SYSTEMS AT DIFFERENT STAGES OF THEIR DEVELOPMENT. Vehicle Requirement Actual Autosub3 n/a.75 Predator RQ-1A (concet demonstrator) n/a.4 Predator RQ-1B (early roduction).8.93 Pioneer RQ-2A ( ) Pioneer RQ-2B Hunter RQ-5 (reliability enhanced ) Average UAV The Autosub3 availability is slightly lower than the average of the US military UAVs listed; considerably below the.98 of the Hunter RQ-5, but almost twice that of the Predator RQ-1A when it was a concet demonstrator. As Autosub3 is a unique build a comarison with a concet demonstrator is not out of lace. A reasonable availability target for a unique or low-volume build AUV would be.8 based on these results, which could be achieved through analysis of fault and incident data and remedial

15 15 engineering work. From this analysis, the transition robability 12 is the lowest robability transition in the sequence D Dr O Dv Sh U, and is consequently the area most likely to result in imrovement through a more rigorous testing rocess An increase of 7% in 12 would raise P(D Dr O Dv Sh U) to.8. B. Recovering an AUV Recovering the vehicle entails many risks, and is a mission hase for which some insurers require coinsurance [1]. Understanding and quantifying risk during this hase is therefore imortant. Once on the surface and in range, in the case of Autosub3, a line from the vehicle is graled and the vehicle brought closer to the vessel. Collision with the vessel is one otential risk. The first failure on Autosub3 mission 43 gives another examle of how roblems can occur on recovery (Aendix I, Table VI). A high sea state can add comlications to this hase. For a long range AUV it is not unusual for a mission to take 24 hours and for sea conditions to worsen during the mission. The Markov formalism also allows us to comute the robability of having to recover the vehicle for all the sequences, in actuality, there are two subsets. The first subset comrises those instances where there is a need to recover the vehicle given it has reached the receding state. These are simly single elements from Table III, thus the robabilities of needing to recover immediately following hase: Overboard =.9; Post-dive =.35; Holding/test attern =.19; Underway =.97. The low robability of needing to recover immediately the vehicle is ut overboard is understandable, as few, if any, further tests will have been made at that stage. Desite the fact that a higher number of tests take lace during the holding attern hase, history has shown that failures tend to manifest themselves during the diving hase, thus there is a higher robability that the vehicle will be recovered during the diving hase than during holding attern hase. Recovery is most likely after the end of a mission. The second subset examines the robability of needing to recover the vehicle over a san of two or more states, or via two or more routes. For examle, in the question raised in the introduction, the oerator is keen to know the robability of having to recover a vehicle once overboard if it is not able to set out on its mission. That is, what is the robability of reaching R via either of O, Dv or Sh?

16 16 1. Recovery from Overside? [D Dr O R] =.74; 2. Recovery from Dive? [D Dr O D v R] =.279; 3. Recovery from holding attern? [D Dr O D v S h R] =.145; 4. The robability of having to recover the vehicle following hases O, Dv or Sh is.498. C. Surviving the deloyment Once the vehicle is underway the robability of losing the vehicle is calculated using distance-related survival statistics only [7]. However, a mission comrises several non-underway hases and the benefit of our aroach is that it rovides an estimate of the robability of the vehicle surviving each hase of the deloyment. The successful comletion of a mission (failure free mission) results in an increase in vehicle reliability, hence the transition robabilities need to be udated. Likewise if a mission resents one or more failures, these failures need to be added to the risk model and the transition robabilities must also be udated. However, the Markov chain model here resented is static, in that it allows us to estimate the risk associated with one mission only. A science camaign usually entails several missions [16], hence the model needs to be udated every time a new mission is entered. A successful mission corresonds to a direct sequence D Dr O Dv Sh U R Dr. Using the Markov assumtion P (D Dr O Dv Sh U R Dr) = =.72. That is, rior to the start of the on-board testing, the oerating calling on the use of Autosub3 can exect a.72 robability of recovery after a successful mission. This is the troublefree sequence. Of course, the overall robability of loss is not or.38, as Availability (.75) is a major factor. The robability of losing the vehicle immediately following hase: Overboard =.15; Post-dive =.85; Holding/test attern =.1; Underway =.3. A key question is, What is the ath that is more likely to lead to the loss of the vehicle? The robability for each sequence is resented below: 1. Loss from Overside? [D Dr O L] = Loss from Dive? [D Dr O Dv L] =.678.

17 17 3. Loss from Holding? [D Dr O Dv Sh L] = Loss from Underway? [D Dr O Dv Sh U L] =.224 (for an 8km mission). Aart from when the vehicle is underway, in oen water oerations the most significant risk of loss is when the vehicle is in diving mode. D. Decommissioning the vehicle and estimated working life Scraing the vehicle is a ossibility considered by the AUV oerators following the salvage or finding of the vehicle. Scraing or recycling can result in additional costs and the robability of this haening can be estimated using the Markov aroach. This state can be reached from any of seven starting oints, Table V. TABLE V PROBABILITIES OF NEEDING TO SCRAP THE AUV GIVEN THAT THE VEHICLE WAS PREVIOUSLY IN STATES DR, O, DV, SH, U, L OR R. From state P(Scraed) Dr.5 O.12 (S).15 (F) Dv.84 (S).7 (F) Sh.545 (S).453 (F) U.297 (S).248 (F) L.99 (S).825 (F) R.198 (S).165 (F) NOTE: The suerscrit (S) indicates a ath following Salvage and suerscrit (F) indicates a ath following vehicle Found. There is.1815 ( ) robability of having to scra the vehicle given that the vehicle has been lost. This comes as no surrise. More interestingly is that even though the ath [Dr -> Sc] is shorter than the ath [U -> Sc]. The robability of having to scra the vehicle is lower if the deloyment is at state Dr; the robability of having to scra the vehicle given that the deloyment is at state U is.545 ( ). In robability theory, Markov chains are tyically studied by considering secial tyes of Markov chains, these are defined according to their toology. The node state Sc (vehicle scraed) is denoted in the literature as an absorbing state. This is due to the fact that it is not ossible to jum from state Sc to

18 18 any other state, all arrows are converging to state Sc, there are no arrows leaving state Sc. A Markov chain that contains one or more absorbing nodes is called an absorbing Markov chain [14]. The robability of reaching an absorbing state will always increase with the number of transitions until the rocess is totally absorbed by this state. In ractical terms, and with resect to the model resented in this aer, the number of transitions to total absortion is an indication of the likely working life of the vehicle. This estimate is resented in the fundamental matrix of the absorbing Markov chain. The rocess to follow to derive the fundamental matrix is resented in the literature [14]. The fundamental matrix is resented in 5, where values are aroximated to the three most significant digits N The first line in the fundamental matrix informs us that assuming that the rocess starts in state 1, on average the rocess would ass 94.5 times in state 1, 82.7 times in state 2, 79.3 times in state 3 and so on for the remaining states, before being absorbed by state Sc. In this case, using the same vehicle, with no maintenance or relacement of comonents, the vehicle would be able to carry out 72 underway missions before it ended u scraed. At first, this figure may seem essimistic, after all Autosub2 was lost after 216 successful missions. To assess the significance of this estimate, one should consider Autosub3 fault history to date. The risk model resented in this aer has been develoed based on oerational history sulied from Autosub3 missions 384 to 422. Autosub3 has comleted 12 more missions since mission 422, bringing the total number of missions to 5 (to February 29). Half way through mission 431, Autosub3 collided with the ice shelf, whilst under the Pine Island Bay Glacier, Antarctica. This high imact fault caused structural damage to the vehicle. Autosub3 survived mission

19 and managed to comlete two more missions under ice shelf. However following the incident that occurred during mission 431, a grou of exerts from NOCS, with a combined exerience of 55 years, were asked to assign a robability of losing the vehicle given that the same fault emerges under the same conditions. The aggregated robability judgment was.58; Autosub3 was fortunate to have survived its 47 th mission, mission 431 [25]. Thus, in light of this recent event, the estimate of 72 underway missions before being scraed aears to be more lausible. V. DISCUSSION The roosed Markov model rovides a useful aroach for estimating the risks during hases in an AUV deloyment. Its grahical structure injects transarency into the AUV deloyment rocess that facilitates rocess criticism and imrovement for each hase. In the motivating examle resented, the model transition robabilities were based on exert judgments and statistical survival analysis of Autosub3 fault and incident history. Examles of how the model can be used to address questions concerning relative and absolute risk relevant to the owner, managers, and engineers have been resented. Analysis showed that the model roduced lausible answers to all these queries. It would be feasible to add time as a covariate in this analysis. Time is often at a remium at sea, and questions over how long vehicle tests may take, or how long might recovery take after a failed deloyment, are not uncommon. Sojourn time at each state has not been modelled here, the data simly did not exist. Future work seeking to address this roblem would need to draw uon more detailed record keeing. There is a case for a common form of structured record keeing for AUV deloyments, informed by this tye of hase toology, so that individual and comarative statistics can be obtained. The integration of the Markov model with distance-based statistical survival models and the use of such models to estimate the risk of a science mission is novel, and as shown, this aroach can rovide more detailed risk estimates. Verification of the absolute and relative robabilities is difficult, the first stage, though feedback of the results to those from whom judgments have been elicited, and subsequent revision to robability

20 2 estimates, was undertaken in this work. A second stage might comare the results with one AUV to the results from another tye. For those hases where a frequentist aroach would be valid, direct comarison of judgments and recorded frequency would be informative. Arguably, the results obtained with a revious Autosub3 statistical model [16] cannot be used to validate the results roduced by this Markov chain model. Only a fraction of the failures considered reviously were incororated for the underway asect of the Markov model, furthermore additional exert judgments were needed to oulate the non-underway transition robabilities. Lastly, the vehicle s configuration is likely to introduce constraints on the reliability and subsequent changes to mission risk. Maturity of the vehicle s configuration will also influence mission risk. This henomenon is not catured in the roosed aroach. It is clear from UAV reliability analysis that risk may decrease markedly between a concet demonstrator and a roduction vehicle [24]. Future work should seek to model how different AUV configurations and maturity may influence the oerational risk.

21 21 Aendix A TABLE VI TABLE OF MISSIONS, DISTANCES, FAULTS WITH COMMENTS AND STATE CLASSIFICATIONS No. Distance km Fault/incident descrition Transition Mission aborted (to surface) due to network failure. (Much) later tests showed general P47 roblem with the harnesses (bad crim joints). Loo of recovery line came out from storage slot, long enough to tangle roeller. P Autosub headed off in an uncontrolled way, due to a side effect of the removal of the P57 uwards-looking ADCP GPS antenna failed at end of mission. P Homing failed, and the vehicle headed off in an uncontrolled direction. Mission was P87 stoed by acoustic command. Problem was due to (a) the uncalibrated receiver array, and (b) a network message ( homing lost ) being lost on the network Aborted after 4 minutes ost dive, due to network failure. Logger data showed long P67 gas, u to 6s, across all data from all nodes, suggesting logger roblem. Deth control showed instability. +/- 1m oscillation due to incorrect configuration gain P87 setting Vehicle went into homing mode, just before dive and headed north. Vehicle mission P57 stoed by acoustic command. It was fortunate that the shi-side acoustics configuration allowed the shi to steam at 9kt (faster rather than 6kt with the towfish) and catch the AUV. Searately, homing mode not exited after 2 minutes, as exected. It will continue on P87 last-determined heading indefinitely a Mission Control configuration error. Problem with deck side of acoustic telemetry receiver front end, unrelated to vehicle P87 systems ADCP down range limited to 36m, reduced accuracy of navigation. P67 GPS antenna flooded. No fix at end oint of mission. P67 EM2 swath sonar stoed logging during mission. P As consequence of GPS failure on M391, AUV ended u 7m N and 25m E of P87 exected end osition Acoustic telemetry giving oor ranges and no acoustic telemetry. P Jack-in-the-box recovery float came out, wraing its line around the roeller, P87 jamming it, and stoing the mission. Caused severe roblems in recovery, some damage to uer rudder frame, sub-frame and GPS antenna. Required boat to be launched Jack-in-the-box line came out, wraed around the roulsion motor and jammed. P Current estimation did not work, because minimum time between fixes for current to P67 be estimated had been set to 15min; leg time was only 1min. Mission stoed and restarted with configurable time set to 5min Main lifting lines became loose, could have jammed motor. P Oerators ended mission rematurely, they believed the AUV was missing wayoints. In fact, a coule of wayoints had been ositioned incorrectly. P67

22 22 TABLE VI(cont.) No. Distance Fault/incident descrition Transition km Configuration mistake; ADCP u configured as down- looking ADCP causing P67 navigation roblems through tracking sea surface as reference. This data was very noisy and ut vehicle navigation out by a factor of 1.5. Damaged on recovery, moderately serious to sternlane, shaft bent. P Stern Plane stuck u during attemt to dive, 2d 2h into mission. Stern lane actuator P37 had flooded. Abort due to network failure. Abort release could not communicate with deth P67 control node for 43s. Possibly side-effect of actuator or motor roblems. Motor windings had resistance of 33 ohm to case. Proeller seed droing off P67 gradually during a dive Only one osition fix from tail mounted ARGOS transmitter. P87 GPS antenna damaged on recovery. P Recovery light line was wraed around the roeller on surface. Flas covering the P87 main recovery lines (and where the light line was towed) were oen. Took over 1 hour to get GPS fix at final wayoint. P67 Proeller seed showed same roblem as m42. Subsequent testing of motor with P67 Megger showed resistance of a few kohm between windings Pre-launch, abort weight could not be loaded successfully due to distorted keeer. If P21 not sotted, could have droed out during mission, considered low robability of distortion and not checked. Pre-launch, otential short circuit in motor controller that could sto motor. P21 Proeller seed showed same roblem as on m42 and 43. P21 CTD dro-out of 1 hour (shorter dro-outs noted in revious missions). P67 M44 recovery was comlicated when lifting lines and streaming line became traed P87 on the rudder (robably stuck on the Bolen where the two were attached). Recovery from the situation required the traed lifting lines graled astern of the shi, attached to the gantry lines, and the caught end cut. The forward sternlane was lost due to lifting line traing between the fin and its P87 fla on recovery. The acoustic telemetry nose transducer was damaged due to collision with the shi. P Fault found re-launch, LXT tracking transducer had leaked water relaced. P21 Fault found re-launch, starboard lower rudder and sternlane loose. P AUV ran slower than exected and seed droed off during mission, due to motor P67 roblem. Current sikes of 3A and voltage dros in first art of mission. P67 Proulsion motor failed 5V Megger on recovery on windings to case. P87 One battery ack out of four showed intermittent connection. P67 Acosutci telemetry unit gave no relies. P87 On surfacing first GPS fix was 1.2km out. P87 Sikes in indicated motor rm P Acoustic telemetry unit gave no relies at all no tracking or telemetry. P57 Noise sikes on both channels of turbulence robe data. P67

23 23 Table VI(cont.) No. Distance km Fault/incident descrition Trans ition Proulsion motor felt rough when turned by hand bearings relaced before P21 deloyment. Aborted at 5m due to overdeth as no deth mode commanded. Unless comounded P36 by another roblem, this would show itself immediately on first dive. No telemetry from Acoustic telemetry unit. P57 Difficulty stoing Autosub on surface via radio command. Searate roblems with the P57 two WiFi access oints. Still sikes on motor rm that need investigating. P No acoustic telemetry or transonding. LXT shi side USBL receiver had leaked P57 during mission giving oor bearings to sub, relaced with sare No acoustic telemetry or transonding. P No GPS fix at the end of the mission. GPS antenna bulkhead had water inside and had P87 flooded No GPS fix at end of mission. After next mission, GPS fixes started coming in after P87 vehicle ower u/ower down; erhas roblem was due to initialisation with receiver and not this time the antenna. Problem at start for holding attern. Holding attern timed out due to rogramming P57 mistake Prior to dive, checks showed reduced torque on rudder actuator. Actuator relaced with P57 new one - first use for this new design of actuator motor and gearbox. However, AUV sent most of mission stuck going around in circles at deth due to rudder actuator fault. The new actuator overheated, melting wires internally, the motor seized, and internal to the main ressure case, the ower filter overheated. Some of the damage may have been caused by an excessive current limit (3A); correct setting was.3a. But this does not exlain high motor current. Possible damage during testing when motor stalled on end sto? Comounded by wiring to motor held tightly to case with cable ties, and worse, covered with tae (acting as an insulator). Wires were not high temerature rated. Three harness connectors failed due to leakage, affecting ayload systems: EM2 P67 tube, ADCP_down, and Seabird CTD. Desite connector roblems the system worked without glitches and failed only when the ower ins had burned comletely through on the connector feeding ower to the abort system Although it worked roerly at the start of the mission at a range of 12m, the P57 acoustic telemetry stoed working at the end of mission. Hence could not sto the mission acoustically when needed Not ossible to communicate with vehicle at 118m deth; holding attern caused a P57 timeout, and AUV surfaced. Acoustic telemetry max range was 5m for digital data When homing was stoed deliberately after 1 min, the AUV did not go into a stay here mode. Rather it continued on the same heading; stoed by acoustic command 5m from shore. Cause was incorrect configuration of mission excetion for homing. Default in camaign configuration scrit was not set due to inexerience with new configuration tools. P87

24 24 ACKNOWLEDGMENT We are grateful to the ten AUV exerts from outside the UK that gave freely of their time in assessing the fault history of Autosub3. Their work rovided the baseline robabilities without which this work would not have been ossible. We are esecially grateful to the Autosub technical team for their source data on faults and incidents, we are articularly grateful to Mr. Steven McPhail and Mr. Peter Stevenson for roviding the judgments that heled oulate most of the transition robabilities resented in this aer. REFERENCES [1] Griffiths, G., Bose, N., Ferguson, J. and Blidberg, R., 27. Insurance for autonomous underwater vehicles. Underwater Technology, 27(2): [2] Griffiths, G., Millard, N.W., McPhail, S.D., Stevenson, P. and Challenor, P.G., 23. On the reliability of the Autosub autonomous underwater vehicle. Underwater Technology 25, (4), [3] Podder, T.K., M. Sibenac, H. Thomas, W. Kirkwood, and J.G. Bellingham, 24. Reliability growth of autonomous underwater vehicle Dorado. In: Proceedings of the Marine Technology Society/Institute of Electrical and Electronics Engineers Oceans Conference, Kobe, Jaan, [4] Griffiths, G. and Brito, M., 28. Predicting risk in missions under sea ice with Autonomous Underwater Vehicles. In, Proceedings of IEEE AUV28 Worksho on Polar AUVs [CDROM]. Richardson TX, USA, IEEE. [5] Griffiths, G. and Brito, M.P., 29. A Bayesian Aroach to Predicting Risk of Loss During Autonomous Underwater Vehicle Missions. IEEE Journal of Oceanic Engineering, submitted for ublication.

25 25 [6] Griffiths, G. and Trembanis, A., 27. Towards a risk management rocess for autonomous underwater vehicles. In, Griffiths, G. and Collins, K. (eds.) Masterclass in AUV Technology for Polar Science: London, UK, Society for Underwater Technology, [7] Brito, M. and Griffiths, G., 28. Using Exert Judgments on Autonomous Underwater Vehicle Probability of Loss to Estimate Vehicle Survivability in Four Oerating Environments. Risk Analysis, submitted for ublication. [8] Matlab, 29. Mathworks:Matlab version 12. Available online: htt:// [Accessed 26 May 29]. [9] Grimes, J.D., 197. On determining the reliability of rotective relay systems. IEEE Trans. Reliab., R-19, [1] Alam, M. and Al-Saggaf, U., Quantitative Reliability Evaluation of Reairable Phased- Mission Systems Using Markov Aroach. IEEE Trans. Reliab., R-35 (5), [11] Whittaker, J.A. and Thomason, M.G., A Markov Chain Model for Statistical Software Testing. IEEE Trans. Softw. Eng., 2(1), [12] Furukawa, K., Cologne, J.B., Shimizu, Y. and Ross, N.P., 29. Predicting Future Excess Events in Risk Assessment. Risk analysis, 29(6), [13] Guanquan, C., and Jinhua, S, 29. Quantitative Assessment of Building Fire Risk to Life Safety. Risk Analsysis, 28(3),. 615:626. [14] Feller, W., 195. An Introduction to Probability Theory and its Alications, vol. 1. New York: John Wiley and Sons. [15] Ouhbi, B., and Nikolaos Limnios, Reliability estimation of semi-markov systems: a case study. Reliability Engineering and System Safety, 58, [16] Brito, M., Griffiths, G. and Trembanis, A., 28. Eliciting exert judgment on the robability of loss of an AUV oerating in four environments. Nat. Oceanograhy Centre Southamton, Research and Consultancy Reort 48, 28. [Online]. Available: htt://erints.soton.ac.uk/54881/.