Robotics and Computer-Integrated Manufacturing 27 (2011) 4755

Contents lists available at ScienceDirect

Robotics and Computer-Integrated Manufacturing

journal homepage: www.elsevier.com/locate/rcim

Learning IFTHEN priority rules for dynamic job shops using genetic algorithms
Muzaffer Kapanoglu a,n, Mete Alikalfa b
a b

Osmangazi University, College of Engineering, Department of Industrial Engineering, Eskisehir, Turkey Osmangazi University, Department of Informatics, Eskisehir, Turkey

a r t i c l e in fo
Article history: Received 9 January 2008 Received in revised form 31 May 2010 Accepted 1 June 2010 Keywords: IFTHEN priority rule State dependent priority rule Queue intervals Genetic algorithm Machine learning Dynamic job shops

abstract
Performing complex, informed priority rules might pose a challenge for traditional operator-driven systems. However, computer-integrated manufacturing systems may signicantly benet from the complex, informed rules such as state-dependent priority rules. A state-dependent priority rule can be dened as a list of IFTHEN priority rules that will be performed if certain system conditions are satised. Here, we propose a genetic algorithm based learning system for constructing interval-based, state-dependent priority rules for each interval of queue lengths in dynamic job shops. Our approach builds interval based state-dependent priority rules pairing the priority rules with the intervals of queue lengths, and determines priority rules and their corresponding length of intervals for a given objective. A genetic algorithm is developed for matching queue length intervals with appropriate priority rules during simulation. A system simulation evaluates the efciencies of interval based state dependent priority rules. The experiments show that interval-based state dependent priority rules obtained by the proposed approach considerably outperform the priority rules including shortest processing time (SPT), minimum slack time (MST), earlier due date (EDD), modied due date (MDD), cost over time (COVERT), and critical ratio (CR) for total tardiness for most of the problems. & 2010 Elsevier Ltd. All rights reserved.

1. Introduction Scheduling is a very important task for companies in todays competitive markets and ever increasing number of research efforts have investigated the solution of the scheduling problems. Nevertheless, the scheduling problems cannot be solved optimally for even modest problem sizes. Studies in manufacturing scheduling mostly deal with priority rules without any consideration to the states of the system sometimes due to ease of use in the shop oor. In traditional manufacturing systems, scheduling is carried out by machine operators and shop supervisors. Therefore complex, multiattribute, or state-observing rules can cause a serious surveillance problem. Computer-integrated manufacturing (CIM) systems on the other hand do not face this sort of challenges. In this article, we introduce an interval-based learning scheduling system for building state-dependent priority rules in CIM scheduling. Due to their ease of implementation and, again, their reduced computational requirement, priority rules (aka scheduling and dispatching rules) have been popular approaches for a long time.

Corresponding author. Tel.: +90 222 239 3750x3246/3247; fax: +90 222 239 3613. E-mail addresses: muzaffer@ogu.edu.tr, mete@ogu.edu.tr, mkapanoglu@gmail.com (M. Kapanoglu). 0736-5845/$ - see front matter & 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.rcim.2010.06.001

A priority rule is used to determine which job from a queue is to be operated next. Numerous priority rules have been introduced in the literature. Lawrence [1] compares the performance of ten individual priority rules with a randomized combination of these rules, and shows that the combined method provides far superior results but requires substantially more computing time. No priority rule appears to be superior to all the others [2]. Their efciency depends on the performance criteria of interest and the operating conditions (e.g., attributes of jobs/machines). Consequently, the best priority rule cannot be known a priori many a times. Some researchers adopted simulation to determine the most favourable priority rule(s) for the operating conditions, the production objectives, and the current shop status [3]. Pierreval and Mebarki [2] developed a heuristic dispatching strategy, called shift from standard rules (SFSR). The strategy proposed is based on a dynamic selection of certain predetermined priority rules. The method for the selection of priority rules includes an optimization algorithm for the numerical thresholds obtained via simulation. Some researchers propose meta-heuristics such as genetic algorithms and more recently, articial intelligence (AI) methods. Most AI approaches to the scheduling problem have been expert system approaches or heuristic search methods combined with expert knowledge [4,5]. Without an adequate learning mechanism, expert system approaches are perceived as unsuitable for scheduling problems [6]. Over the last two decades, only a limited number

48

M. Kapanoglu, M. Alikalfa / Robotics and Computer-Integrated Manufacturing 27 (2011) 4755

of research articles seem to have addressed the use of machine learning approaches for manufacturing scheduling problems among those surveyed, especially on those with state-dependency. Methods used in these applications include rote-learning, neural networks, induction, and case-based reasoning [7]. Dorndorf and Pesch [8] conducted a probabilistic learning approach, where each gene represents one rule from out of a set of decision rules. The decision rule at the ith position means that a conict in the ith iteration should be resolved by using that rule. The algorithm searches for the best sequence of decision rules for selecting operations to guide the search of a heuristic scheduling algorithm. Hilliard et al. [9] rst applied classier systems (i.e., genetics-based machine learning) to discover general rules for job shop scheduling. The learning objective is to learn the order of jobs in a queue. Caskey and Storch [10] have shown that allowing each machine to select their priority rule independently (as they call heterogenous dispatching rule) might have advantages. Lee et al. [11] applied genetic algorithms to dispatch jobs at each machine by inducing decision trees, a machine learning technique, to release jobs into shop oor. Aytug et al. [12] consider the automated learning of strategies for real-time scheduling in dynamic factory oor environments by using Michigan style GBML. In Aytugs study, learning is used to update the knowledge bases of intelligent dispatchers in the oor shop. Priore et al. [13] emphasize the importance of state-dependent priority rules. They propose a supervised learning system to generate heuristic rules for selecting the priority rules based on the state of the system in a dynamic environment. Tamaki et al. [14] developed a priority equation to set priorities of jobs by using genetics-based machine learning (GBML). Aytug et al. [7] provide a comprehensive review of learning methods in manufacturing scheduling. This paper introduces an unsupervised learning scheduling system that builds state and priority rule pairs depending on intervals of queue by using a rule-based genetic algorithm for dynamic job shop scheduling. The number of priority rules produced for a machine equals the maximum number of intervals in queue. Therefore, in our system, each machine can independently execute the proposed priority rule for that state of its own queue using a common state-priority rule list. Machines in the system collaborate in learning which rule for which interval in queue but once the list that matches the rules with states is built, then all machines use the provided common state-priority rule list. This common list eases the application of the approach especially for semi-automated manufacturing operations since no machine and part properties are considered except number of intervals and intervals length in queue. The objective value (i.e. total tardiness) reached by that specic set of interval based state-dependent priority rules is computed by system simulation. Nine sets of problems, each with 5 problems, are generated based on varying tightness of due dates (similar to Aydn and Oztemel [15]). We used minimizing total tardiness as objective. The paper is organized as follows. Section 2 covers denitions and discussions related to dynamic job shop scheduling and priority rules. Section 3 describes genetic algorithms as pertaining to machine learning. The proposed system is discussed in Section 4. Computational experiments are presented in Section 5. Interpretation of experiments is presented in Section 6. The last section includes the conclusion and future research issues.

machine for a certain length of time (called processing time) pjk. Jobs may not require all m machines, and may have to visit some machines more than once. A due date dj may be assigned to each job Jj as well. Each job has an arrival time and a due date. The arrival time is the time by which the job can start its processing. And the due date is the time by which the job has to be completed. A dynamic job-shop scheduling (DJSS) problem is dened as determining the sequence of jobs before machining operations according to a given objective. In a dynamic JSS problem, unlike static job shops, jobs can arrive at some known (deterministic) or unknown (stochastic) future time.

2.1. Priority rules The priority rules, also called dispatching or scheduling rules, have been widely used to provide good and time-efcient solutions to job-shop scheduling problems for decades [16,17]. In this study, nine, most preferred priority rules for tardiness are collected from the previous studies to investigate the contribution of state-dependent priority rules [15,18,19]. These nine rules select the job with minimum Zij as the next job to be processed on the machine, where Zij is the priority index, and is dened differently for each rule. Table 1 summarizes these nine rules, their priority equations and notations. During the last 30 years, the performances of these rules have been studied extensively using varying techniques including simulation. These studies have addressed determining the best rule for different problem settings. At the present time, there is no priority rule that overrules the others [2,16] under the varying operating conditions (e.g., tightness of due dates, shop loading rate, etc.) even for a single performance criterion. Any priority rule may turn out to be inefcient due to changes in the operating conditions. Kapanoglu and Alikalfa [20] have shown that statedependent priority rules obtained from genetic algorithm based machine learning outperform priority rules. In their approach, considering the fact that the adopted priority rule may turn out to be inefcient due to any change in the operating conditions, they address issues of (i) which priority rule can yield better performance and (ii) what changes in the operating conditions lead to adopting a new rule(s) in the context of general job-shop. When it comes to dealing with dynamic job-shops, the practicality of the schedules generated should be considered as third
Table 1 The priority rule list with priority equations. Name FIFO (rst n rst out) EDD (earliest due date) MST (minimum slack time) SOPN (minimum slack time per operation) SPT (shortest processing time) CR (critical ratio) SRPT (shortest remaining processing time) MDD (modied due date) COVERT (cost over time) Notations rij roi di pij rti kb t arrival time of job i at machine j total remaining operations of job i due date of job i processing time of job i at machine j remaining processing time of job i parameter of the COVERT rule current time Zij rij di di rtit (di rti t)/roi pij (di t)/rti rti P max{di,ti + pij} (1/pij)(1 s + /(kb(rti pij))) + (the larger, the higher priority)

2. Dynamic job shop scheduling In a dynamic job shop, there are n jobs Jj (j 1,y, n) to be processed on m machines Mi (i 1,y, m) for operations Ojk (k 1,y, nj). Each operation is performed by a predetermined

M. Kapanoglu, M. Alikalfa / Robotics and Computer-Integrated Manufacturing 27 (2011) 4755

49

issue excluding full-edged computer integrated manufacturing systems where state dependent priority rules can easily be implemented. In this study, our proposed system resolves these issues effectively. From the standpoint of better performance, the proposed system determines interval-based state dependent priority rule that cannot be beaten by a single priority rule as given in Lemmas 1 and 2. Lemma 1. A priority rule corresponds to an instance of an intervalbased state dependent priority rule. Proof. Interval-based state dependent priority rule consists of rules that will be applied for varying number of jobs in queue of a machine. Therefore, using the same priority rule regardless what the number of jobs in queue corresponds to single priority rule as widely practiced. & Lemma 2. For a given single performance criterion, an optimal interval-based state dependent priority rule cannot be outperformed by an optimal single-priority rule. Proof. Following Lemma 1, if a single-priority rule outperforms the interval based state dependent priority rule and is optimal for a given performance criteria, then the interval-based state dependent priority rule will converge to that specic instance within the experimental conditions, i.e., choosing the same priority rule for each interval. & For relating the shop conditions to the priority rules that should be selected, there exist numerous operating conditions such as machine queue lengths, system and/or machine utilization, workin-process, etc. [11,12,21]. We adopted the intervals of queue lengths of machines at a specic point of time as the key representative of the operating conditions. The proposed system observes the interval of queue lengths of machines, and learns a common best rule-set where each rule is learned to be best for a corresponding interval of queue lengths. For practicality, we seek state dependent priority rule that prefers larger intervals of queue lengths from out of alternatives with the same or similar efciencies. We adopted the total tardiness as the performance criterion in the experiments we worked out.

unsuccessful CA rules. The strength measures the inuence of a particular CA rule in an evolutionary process, driven by a GA. GA seeks to improve the systems overall performance by replacing unsuccessful CA rules with successful ones. These CA rules are evaluated in an environment, and are automatically assigned strengths based on systems performance. As a result, GBML learns by interacting with an environment from which it receives feedback in the form of numerical reward. Learning is achieved by generating the CA rule set that maximizes the amount of reward received.

4. Proposed interval-based genetic algorithm Our interval-based learning scheduling system consists of two main parts: genetic algorithm (GA) and simulated environment (SE). GA is responsible for constructing interval-based, statedependent priority rules which contains a list of priority rules that are associated with different intervals of queue lengths. GA evaluates each chromosome to determine their performances through SE. Chromosomes survive according to their performances. SE is responsible for simulating the scheduling process dynamically by using state-dependent priority rule whenever it needs to select a job to assign to any machine. Fig. 1 represents the interval-based learning scheduling system briey. Whenever any assignment event happens, the machine uses the state-dependent priority rule to identify an appropriate priority rule. In our system, each machine can independently execute the appropriate priority rule for that state of its own queue using a common state-priority rule list. Machines in the system collaborate in learning which rule for which interval in queue but once the list that matches the rules with states is built, then all machines use the provided common state-priority rule list. In the case of departure events, if the job is completed it sends its details to the data le; otherwise it puts it into the related queue for the next operation. The assignment and the departure procedures are shown in Fig. 2a and b. The simulation carries on as long as the stopping criterion is unsatised. In this case, the simulation stops when the predened number of jobs is scheduled. In the proposed system, no machine or workstation has a predetermined priority rule, rather a set of rules where each rule is learnt to be the best for a given interval of queue length and performance criterion. Priority rules that will be applied for specic intervals of queue lengths along with interval widths are learned by the scheduling system.

3. Genetic algorithm based machine learning Genetic algorithms (GA) are based on Darwins theory of evolution. By starting with a set of potential solutions and changing them over generations, GAs converge to the most t (optimum or near optimum) solution [22]. Extending GA further, genetics-based machine learning (GBML) is an automated learning paradigm to discover condition-action (CA) rules for performing desired actions. In contrast to traditional expert systems where rules are handcrafted by knowledge engineers, GBML use genetic algorithms as discovery algorithm to generate and develop CA rules. The CA rules contain two parts; condition part and action part as presented below. IF o condition 4 THEN o action 4 Whenever the condition part of a CA rule matches the current state of the environment, (i.e., the CA rule is satised) then the action part of the CA rule is performed (i.e., the CA rule is red). That is, CA rules match information about the current state of system and suggest actions, and system moves to a new state. Separating successful CA rules from less successful (or unsuccessful) CA rules, an evaluation (or reward) mechanism needs to be implemented. Therefore, strength Si(t) is associated with each CA rule i at a time t. The strength is modied by a certain reward mechanism, increasing the strength for successful CA rules, and decreasing the strength for

4.1. Representation of chromosome In the proposed approach, a set of state-dependent priority rules is represented as a chromosome including a set of n priority rules, R {R1,R2,y,Rn} and a set of n intervals of queue lengths, L1 L2 Ln I {I1 , I2 ,y, In } where Li is a length of the ith interval. The designed chromosome contains the priority rules and the corresponding intervals. In the designed chromosome, there is a specic priority rule for each interval of queue lengths such as MST for interval [1,12], and FIFO for interval [13,17], and SPT for [max_queue_length-2, max_queue_length] for a manufacturing system with maximum jobs appeared in queues of machines at any point of time in the scheduling period (max_queue_length). The chromosome presented in Fig. 3 is interpreted below where QL is the number of jobs in the queue at any moment. IF 1 r QL r 12 THEN MST IF13 r QL r 17 THEN FIFO ... IFmax_queue_length2 r QL r max_queue_length THEN SPT

50

M. Kapanoglu, M. Alikalfa / Robotics and Computer-Integrated Manufacturing 27 (2011) 4755

START GA

Start Simulation

Initialization Decode chromosome to generate state-dependent priority rule Update fitness value

Determine the details of arrival job Put it into related queue for the first operation Identify the inter-arrival time for the next arrival

Evaluate the fitness of each chromosome using simulation

Selection Crossover

Assignment

Mutation

Any assignment prior to the next arrival?

Is stopping criteria satisfied?

Any departure prior to the next arrival??

Departure

END of GA

Increase simulation time upto the next arrival time

Is stopping criteria satisfied?

End of Simulation
Fig. 1. Interval-based learning scheduling system.

Start Assignment

Start Departure

Select the priority rule to be applied from statedependent priority rule Specify an operation from the queue using the rule Dispatch the operation to the machine n ad calculate the finishing time Remove the operation from the queue and decrease queue length Main Procedure Send the job to related queue for the next operation

Determine the earliest finishing operation Increase simulation time upto the end of the operation Release the machine

Any operation remain on the job?

Store data on the file

Main Procedure

Fig. 2. (a) Sub-procedure of assignment and (b) sub-procedure of departure.

M. Kapanoglu, M. Alikalfa / Robotics and Computer-Integrated Manufacturing 27 (2011) 4755

51

MST

FIFO

SPT

12

Table 3. (continued ) (c) (k 6) FIFO EDD MST SPT SOPN CR SRPT MDD COVERT

Fig. 3. Representation of a chromosome.

Table 2 Parameter settings of proposed interval-based GA. Parameter Population size Crossover rate Mutation rate Number of terations Tournament selection size Value 100 0.65(65%) 0.05(0.5%) 1000 (# of nterval) 4

307.4155 92.8455 390.4002 433.6923 664.7922 799.3869 502.2325 91.1992 247.7673

2925.4713 1403.2559 2102.6625 2656.5041 4366.8171 4737.3144 2230.928 1532.6429 2356.2949

19 155.5301 13 288.2238 11 985.3072 15 190.7363 23 799.0335 26 058.6043 11 220.3102 14 243.9492 16 409.4659

Table 3 Total tardiness values for experiments. Priority rules (a) (k 2) FIFO EDD MST SPT SOPN CR SRPT MDD COVERT 50 jobs 100 jobs 200 jobs

1596.6989 1088.6863 865.1585 1183.5897 1273.1257 1596.0474 1014.5547 1102.5968 1308.4565

8975.7757 6872.2000 5831.0721 6207.5054 7502.4568 9738.6943 5731.6044 7558.4904 6985.2138

34 430.4594 25 926.1688 19 262.4411 24 381.8063 28 930.5726 37 739.7542 18 463.7984 30 812.7651 27 403.9307

Proposed scheduling system Number of intervals 2 71.0281 3 65.3245 4 58.6041 5 57.9929 6 57.5001 7 54.9042 8 54.9042 9 54.9042 10 53.3467 11 51.2495 12 50.3405 13 1416 1728

1393.5067 1128.9726 1032.5007 1030.3603 1012.5652 988.1362 960.3304 951.3524 951.3524 951.3524 951.3524 951.3524 951.3524

10 894.2347 10 581.8106 10 479.4151 10 049.0581 10 049.0581 10 049.0581 10 049.0581 10 049.0581 10 019.9353 10 019.9353 10 019.9353 10 019.9353 7053.9921 7053.9921

The second rule above is interpreted as follows. If there are QL jobs between 13 and 17 in the queue, the next job to be processed at this machine will be selected according to the FIFO rule. 4.2. Operators and tness The operators of our genetic algorithm and the computation of the tness values are described as follows. Selection and reproduction: The tournament selection with the elitist strategy is adopted. In the tournament selection, a small group of individuals (related to tournament size) is sampled from the population, and the individual with the best tness is chosen for reproduction. Elitism appears to eliminate the destruction of the best solution due to selection. In elitism, the ttest member of a population is immunized against the selection and the genetic operations for its survival. Crossover: Two individuals are matched randomly in a population, and the crossover is performed with respect to crossover rate pc. Simple, single-point crossover is used where chromosome of each sibling is obtained by swapping the parts of the parent chromosomes with respect to the crossover point. Mutation: A gene is selected for mutation with mutation rate pm. Our mutation operator changes the rst part of the CA rules (priority rules section) with respect to uniform distribution, and the second part of the CA rules (lengths of intervals section) with respect to uniform distribution from predetermined minimum length of interval to predetermined maximum length of interval, when applied. Fitness: The tness of each chromosome is evaluated based on the schedules obtained by running a simulation. Simulation enables us to compute the objective values for total tardiness.

Proposed scheduling system Number of intervals 2 3 4 5 6 7 8 9 10 1113 1415 1617 18 1924 (b) (k 4) FIFO EDD MST SPT SOPN CR SRPT MDD COVERT 862.6475 854.6051 838.6778 830.9986 830.9986 830.9986 810.5196 810.5196 5392.3961 5392.3961 5392.3961 5352.6185 5352.6185 5315.6317 5294.0904 5294.0904 5294.0904 5294.0904 5294.0904 5294.0904 17 817.9642 17 742.7305 17 720.7779 17 565.0968 17 560.3392 17 560.3392 17 560.3392 17 535.5061 17 535.5061 17 473.6435 17 356.9626 17 266.0695 17 266.0695 17 189.6343

518.6839 192.1205 303.4435 423.0586 681.6693 780.8214 315.7244 192.1205 511.9825

2452.5913 1615.7934 1717.8719 2152.6509 3989.4539 4066.5414 1996.9241 1619.9716 1754.9114

23 132.9468 17 531.4916 15 616.6637 18 171.5081 27 010.6975 32 070.2638 16 099.9527 18 301.348 18 841.2556

Proposed scheduling system Number of intervals 2 183.1344 3 181.1217 4 179.8232 5 178.1697 6 177.8232 7 170.2958 89 170.2958 10 167.3742 1112 165.1066 1317 18 1928

1163.3518 1062.3389 1035.4867 1035.4867 1025.1864 1025.1864 1006.2824 1006.2824 1001.668 1001.668 1001.668

14 077.7927 13 992.6665 13 316.5346 13 316.5346 13 309.2213 13 183.656 13 183.656 13 038.4087 13 038.4087 13 038.4087 12 948.1847 12 948.1847

5. Computational experiments In the experimental analysis, we analyze the effect of the number of interval in the queue for the total tardiness objective. The test problems we generated can be represented as {50, 100, 200}/4/G/TSUM, according to Pinedos n/m/A/B notation [17] where n, m, A and B represent number of jobs, number of

52

M. Kapanoglu, M. Alikalfa / Robotics and Computer-Integrated Manufacturing 27 (2011) 4755

865 845 825 805 183 177 171 165 72 64 56 48 2 3 4 5 18000 17750 17500 17250 17000 14200 13850 13500 13150 12800 10800 10300 9800 9300 8800 8300 7800 7300 6800 k=4 k=2 6 7 8 9 10 11 12

k=2

5400 5350 5300 5250 1175 k=2

k=4

1100 1025 950 1400 1275

k=4

k=6

1150 1025 900 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

k=6

k=6

10

12

14

16

18

20

22

24

26

28

Fig. 4. (a) Increase in the number of intervals, decrease in the total tardiness (n 50 jobs). (b) Increase in the number of intervals, decrease in the total tardiness (n 100 jobs). (c) Increase in the number of intervals, decrease in the total tardiness (n 200 jobs).

M. Kapanoglu, M. Alikalfa / Robotics and Computer-Integrated Manufacturing 27 (2011) 4755

53

machines, shop type, and performance measure(s), respectively. Uj is 0 if Cj r dj, and 1 others. The sum of tardiness as formulated below: X Tsum fmax0, tj dj g
j

objective value obtained from the best-performing scheduling rule. The state-dependent priority rules for two and three intervals are presented in Table 5.

where tj and dj mean completion time and due date, respectively. The processing times for each machine are distributed with respect to Uniform [2,9]. Interarrival times between jobs are determined according to an exponential distribution using the following rule: t 20 1 expRN 4:5

6. Interpretation of the experiments We are interested in the inuence of intervals of queue on the solution quality in the paper. Table 3 shows the corresponding results for different due date tightness and different number of jobs. Our interval-based learning scheduling system signicantly outperforms the best priority rules in all cases. In other words, there is no single incidence that state-dependent priority rules generated by interval-based learning scheduling system performed worse than the best priority rule as shown in Table 3ac. Table 4 shows that the average improvements for different due date tightness and different number of jobs. We see from Table 4 that we obtain between 0.29% and 44.80% improvement compared to the best performed priority rules. Even if using two or three interval for queue that the most basic structure of our approach, we obtain between 0.29% and 34.25% improvement. For larger number of intervals we get better solutions at least the same solutions as shown in Fig. 4ac. We show only some state-dependent priority rules in Table 5. When the state-dependent priority rules are examined, the statedependent priority rules contain the best performed priority rule in most cases. When the number of intervals increased, other priority rules nd a position in state-dependent priority rules with best performed priority rule in order to response better to requirements of system in most cases. Furthermore, there are state-dependent rules with the same tardiness value. When these rules are examined, we nd that there are alternative priority rules for the same intervals, and all combination of these priority rules can be found in the state-dependent priority rules (i.e. k 4, job 50, interval 3).

where RN is the random number and t represents the interarrival time [15]. Due dates are identied with the following rule: DD ri k
n X i1

pi

where DD, r, n, k and pi represent due date, arrival time, the number of operations, coefcient of tightness, and processing time of ith operation, respectively [15]. The due dates generated for tightness parameter are k 2, 4, and 6. Also, 50, 100, and 200 jobs are used in the every test problem. Nine sets of problems (three settings of number of jobs x three settings of due date tightness) each with ve problems are used to compare solutions of our interval based scheduling system with best known nine priority rules, i.e. FIFO, EDD, MST, SOPN, SPT, SRPT, CR, COVERT, and MDD. In order to get stochastically signicant results, we perform three independent runs with three different random number seed for each factor combination. Parameter settings of our interval-based GA are given in Table 2. The proposed interval-based GA has been developed in C#, and computations are run on Pentium Mobile 1.6 GHz and 512 RAM. The experimental results are presented for each group of test problems (n 50, 100, and 200) in Table 3 and in Fig. 4ac. The average improvements of state-dependent priority rules are presented for each group of test problems (n 50, 100, and 200) and each due date type (tight, moderate and wide) in Table 4. The improvements are computed based on the average objective values obtained from our state-dependent priority rules, and the
Table 4 Average improvements for each set of problems and each due date type. Tight due date (k 2) # of Int. 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 1924 50 jobs 0.29 1.22 3.06 3.95 3.95 3.95 6.32 6.32 100 jobs 5.92 5.92 5.92 6.61 6.61 7.26 7.63 7.63 7.63 7.63 7.63 7.63 7.63 7.63 7.63 7.63 200 jobs 3.50 3.91 4.02 4.87 4.89 4.89 4.89 5.03 5.03 5.36 5.36 5.36 5.99 5.99 6.49 6.49 6.49 6.90 Moderate due date (k 4) # of Int. 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 1928 50 jobs 4.68 5.72 6.40 7.26 7.44 11.36 11.36 11.36 12.88 14.06 14.06

7. Conclusion In this article, we introduce an interval-based learning scheduling system for building state-dependent priority rules in CIM scheduling. States are dened as intervals of queue lengths, and a state-dependent priority rule for a machine is a list of rules

Wide due date (k 6) 100 jobs 28.00 34.25 35.91 35.91 36.55 36.55 37.72 37.72 37.72 38.01 38.01 38.01 38.01 38.01 38.01 38.01 38.01 200 jobs 9.85 10.40 14.73 14.73 14.78 15.58 15.58 15.58 16.51 16.51 16.51 16.51 16.51 16.51 16.51 16.51 17.09 17.09 # of Int. 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 1920 50 jobs 22.12 28.37 35.74 36.41 36.95 39.80 39.80 39.80 41.51 43.80 44.80 100 jobs 0.69 19.55 26.42 26.57 27.84 29.58 31.56 32.20 32.20 32.20 32.20 32.20 32.20 32.20 32.20 200 jobs 2.91 5.69 6.60 10.44 10.44 10.44 10.44 10.44 10.70 10.70 10.70 10.70 37.13 37.13 37.13 37.13 37.13 37.13

54

M. Kapanoglu, M. Alikalfa / Robotics and Computer-Integrated Manufacturing 27 (2011) 4755

Table 5 State-dependent priority rules for three interval settings. Tightness k2 # of jobs 50 # of ntervals 1 2 3 1 2 3 1 2 3 1 2 3 State-dependent priority rule(s) MST (IF QL r 3 (IF QL r 3 SRPT (IF QL r 3 (IF QL r 3 SRPT (IF QL r 6 (IF QL r 6 MDD Rule 1: Rule 2: Rule 1: Rule 2: Rule 3: Rule 4: (IF (IF (IF (IF (IF (IF

THEN SPT) (IF QL Z 4 THEN MST) THEN SPT)(IF 4 r QL r 5 THEN MST)( IF QL Z 6 THEN SRPT) THEN COVERT) (IF QL Z 4 THEN MST) THEN COVERT) (IF QL Z 4 THEN MST) THEN MST) (IF QL Z 7 THEN SRPT) THEN MST) (IF 7 r QL r 20 THEN SRPT) (IF QL Z 21 THEN SPT) QL r 3 QL r 3 QL r 3 QL r 3 QL r 3 QL r 3 THEN THEN THEN THEN THEN THEN MST) (IF QL Z 4 THEN EDD) MST) (IF QL Z 4 THEN MDD) EDD) (IF QL 4 THEN CR) (IF QL Z 5 THEN MDD) EDD) (IF QL 4 THEN CR) (IF QL Z 5 THEN EDD) MDD) (IF QL 4 THEN CR) (IF QL Z 5 THEN MDD) MDD) (IF QL 4 THEN CR) (IF QL Z 5 THEN EDD)

100

200

k4

50

100

200

1 2 3 1 2 3 1 2 3

EDD (IF QL r 4 THEN SOPN) (IF QL Z 5 THEN MDD) (IF QL r 2 THEN COVERT) (IF QL 3 THEN SPT) (IF QL Z 4 THEN EDD) MST (IF QL r 15 THEN MST) (IF QL Z 16 THEN SPT) (IF QL r 2 THEN MST) (IF 3 r QL r 15 THEN SRPT) (IF QL Z 16 THEN SPT) MDD Rule 1: (IF QL r 7 THEN MDD) (IF QL Z 8 THEN SPT) Rule 2: (IF QL r 7 THEN EDD) (IF QL Z 8 THEN SPT) Rule 1: (IF QL r 2 THEN SPT) (IF 3 r QL r 7 THEN MDD) (IF QL Z 8 THEN CR) Rule 2: (IF QL r 2 THEN SPT) (IF 3 r QL r 7 THEN EDD) (IF QL Z 8 THEN CR) EDD (IF QL r 2 THEN MST) (IF QL Z 3 THEN EDD) (IF QL r 4 THEN COVERT) (IF 5 r QL r 6 THEN SPT) (IF QL Z 7 THEN EDD) SRPT (IF QL r 18 THEN SRPT) (IF QL Z 19 THEN SPT) (IF QL r 9 THEN SRPT) (IF 10 r QL r 11 THEN SPT) (IF QL Z 12 THEN MST)

k6

50

100

200

1 2 3 1 2 3

for each particular state that matches a priority rule with a state. Dynamic job shop scheduling problem with total tardiness are preferred for justication of our approach due to their intrinsically dynamic and challenging nature. The representation in our genetic algorithm encodes states as conditions and priority rules as actions into chromosomes for learning state-dependent priority rules. Our GA evaluates interval-based state-dependent priority rules by running a system simulation. Generic crossover and mutation operators with a tournament selection have been used with no problem-specicknowledge augmentation. A computational analysis has been applied over 9 problemsets to compare the performance of our approach with nine priority rules. Total tardiness objective is analyzed. The results of our proposed method have been impressive by outperforming the selected-best priority rules prominently for each case. Our state-dependent scheduling approach has been built on the queue length-intervals of machines. Our primary motivation for considering interval based scheduling instead of precise queue lengths is to enhance the practical value in industrial applications without diminishing the quality of the schedules. The approach does not utilize an explicit knowledge related to the problem or the objective, which is tardiness in this case. Therefore, any objective can be condently substituted instead. It is worth mentioning that the rules obtained from our proposed approach can also be utilized in the knowledge-base of a scheduling system. Therefore, some research extensions should reveal the opportunities in this direction. One of the major characteristic of this approach is to consider intervals of queue length in nding the best rule-set, and overlook all other attributes related to parts and

machines. While this simplication increases the practical value of the approach, potential contributions of regarding some other system states and part attributes remains to be an open research issue in the context of learning scheduling systems for CIM environments. Another restriction we have imposed was to build a common state-priority rule for all machines instead of independent, individual list for each machine. Relaxation of this restriction may lead to further research issues. In this study, we allowed intervals to oat through the entire chromosome. A predetermined upper and lower bounds of interval lengths must be studied further for improved, efcient learning. References
[1] Lawrence S. Supplement to resource constrained project scheduling: an experimental investigation of heuristic scheduling techniques. Graduate School of Industrial Administration. Carnegie Mellon University; 1984. [2] Pierreval H, Mebarki N. Dynamic selection of dispatching rules for manufacturing system scheduling. International Journal of Production Research 1997;35:157591. [3] Kouiss K, Pierreval H, Mebarki N. Using multi-agent architecture in FMS for dynamic scheduling. Journal of Intelligent Manufacturing 1997;8:417. [4] Alpar P, Srikanth KN. A comparison of analytic and knowledge-based approaches to closed-shop scheduling. Annals of Operations Research 1989;17:34762. [5] Bel G, Bensana E, Dubois D, Erschier J, Esquirol P. A knowledge-based approach to industrial job-shop scheduling. In: Kusiak A, editor. Knowledgebased systems in manufacturing. London: Taylor & Francis; 1989. p. 20746. [6] Yih Y. Trace driven knowledge acquisition for rule based real time scheduling systems. Journal of Intelligent Manufacturing 1990;1:21730. [7] Aytug H, Bhattacharyya S, Koehler GJ, Snowdon JL. A review of machine learning in scheduling. IEEE Transactions on Engineering Management 1994;41:16571.

M. Kapanoglu, M. Alikalfa / Robotics and Computer-Integrated Manufacturing 27 (2011) 4755

55

[8] Dorndorf U, Pesch E. Evolution based learning in a job shop scheduling environment. Computers and Operations Research 1995;22:2540. [9] Hilliard MR, Liepins GE, Palmer M. Machine learning applications to job shop scheduling. In: Proceedings of the rst international conference on ndustrial and engineering applications of articial intelligence and expert systems, 1988. p. 72837. [10] Caskey K, Storch R. Heterogeneous dispatching rules in job and ow shops. Production Planning & Control 1996;7:35168. [11] Lee CY, Piramuthu S, Tsai YK. Job shop scheduling with a genetic algorithm and machine learning. International Journal of Production Research 1997;35:117191. [12] Aytug H, Bhattacharyya S, Koehler GJ. Genetic learning through simulation: An investigation in shop oor scheduling. Annals of Operations Research 1998;78:129. [13] Priore P, De La Fuente D, Gomez A, Puente J. A review of machine learning in dynamic scheduling of exible manufacturing systems. Articial Intelligence for Engineering Design, Analysis and Manufacturing 2001;15:25163. [14] Tamaki H, Sakakibara K, Murao H, Kitamura S. Rule acquisition for production scheduling: genetics-based machine learning approach to exible shop scheduling. In: Proceedings of the Society of nstrument and control engineers (SICE) annual conference 2003. p. 2527.

ztemel E. Dynamic job-shop scheduling using reinforcement [15] Aydn ME, O learning agents. Robotics and Autonomous Systems 2000;33:16978. [16] Blackstone J, Phillips D, Hogg G. A state-of-the-art survey of dispatching rules for manufacturing job shop operations. International Journal of Production Research 1982;20:2745. [17] Pinedo M. Scheduling: theory, algorithms, and systems. 1st ed.. Englewood Cliffs, NJ: Prentice-Hall; 1995. [18] Chiang TC, Fu LC. Using dispatching rules for job shop scheduling with due date-based objectives. International Journal of Production Research 2007;14:324562. [19] Su CT, Shieu YR. Intelligent scheduling controller for shop oor control systems: a hybrid genetic algorithm/decision tree learning approach. International Journal of Production Research 2003;12:261941. [20] Kapanoglu M, Alikalfa M. Genetics-based machine learning to job shop scheduling. In: The fourth nternational symposium on ntelligent manufacturing systems IMS2004; 2004. p. 104856. [21] Bhattacharyya S, Koehler GJ. Learning by objectives for adaptive shop-oor scheduling. Decision Sciences 1998;29:34776. [22] Holland JH. Adaptation in natural and articial systems. Ann Arbor, MI: University of Michigan Press; 1975.