GGSN Monitoring & First Layer Handling

December 9th, 2013 by Doddy W., PSNOS Dept., CNO Div.

Preface

GGSN is the core network node of GPRS system

GGSN functions:
Assign IP address to subscriber
Perform AAA, maintain connection to SGSN, generate charging, etc

Page 2

GGSN Hardware

Ericsson SSR 8020

Huawei UGW9811
PGP-X16
Ericsson M320

Ericsson M120

Page 3

GGSN APN Mapping

Hostname

Page 4

Hardware

Location

GGSNTBS1

Ericsson M120

TB Simatupang, Jakarta

GGSNTBS3

Ericsson M320

TB Simatupang, Jakarta

GGSNTBS4

Ericsson M320

TB Simatupang, Jakarta

GGSNTBS5

Ericsson SSR8020

TB Simatupang, Jakarta

GGSNBRN1

Ericsson M120

Buaran, Jakarta

GGSNBRN2

Ericsson M320

Buaran, Jakarta

GGSNBRN3

Ericsson M320

Buaran, Jakarta

GGSNBRN4

Ericsson SSR8020

Buaran, Jakarta

GGSNPKB1

Ericsson M320

Arifin Ahmad, Pekanbaru

GGSNSBY1

Huawei PGP-X16

Gayungan, Surabaya

GGSN Coverage

Hostname

Page 5

APN Handling

Coverage

GGSNTBS1

- Internet / telkomsel, mms

- internet.r / telkomsel.r , blackberry.net.r,
mms.r (roaming)
- corporate APN

West & Central Java, Kalimantan, East Java,

Balinusra, Sulawesi

GGSNTBS3

internet/telkomsel, blackberry.net, mms

West & Central Java, Kalimantan, East Java,

Balinusra, Sulawesi

GGSNTBS4

internet/telkomsel, blackberry.net

West & Central Java, Kalimantan, East Java,

Balinusra, Sulawesi

GGSNTBS5

blackberry.net, internet/telkomsel (trial)

West & Central Java, Kalimantan

GGSNBRN1

- Internet / telkomsel, mms

- internet.r / telkomsel.r , blackberry.net.r,
mms.r (roaming)
- corporate APN

West & Central Java, Kalimantan, East Java,

Balinusra, Sulawesi

GGSNBRN2

internet/telkomsel, blackberry.net, mms

West & Central Java, Kalimantan, East Java,

Balinusra, Sulawesi

GGSNBRN3

internet/telkomsel, blackberry.net

West & Central Java, Kalimantan, East Java,

Balinusra, Sulawesi

GGSNBRN4

blackberry.net, internet/telkomsel (trial)

West & Central Java, Kalimantan

GGSNPKB1

internet/telkomsel, blackberry.net

Sumatera

GGSNSBY1

blackberry.net

East Java, Balinusra, Sulawesi

GGSN Interface (1 of 2)

Gn
Gi
Ga

Ga

Gx

Gmb

Gx
Gy
Gp
Gmb

GGSN
Gi

Gn

LIG
Gp

LIG
Gy

Page 6

GGSN Interface (2 of 2)

Gn network Gi network

Internet

SGSN

Gn
Network

Gn

Blackberry
Server

GGSN

Corporate
Network
SGSN

Page 7

Physical Topology

2A
3A

Internet
Gate

5A
6A

xe-1/0/0

Gate
Te-4/4

xe-0/0/0

Te-4/3
Te-4/2

Te-7/1
Te-7/2
Te-8/2
Te-8/4
Te-7/3
Te-7/4

Gi Core

GGTBS1

TBS Physical Connection

GGTBS3
xe-0/0/0
xe-1/0/0

GGTBS4
M320

xe-0/0/0
xe-0/0/5
xe-0/0/10
xe-0/0/15

Gi, PBR, NAT,Radius

and Load Balancer
VIPRION1TBS
F5 PB100

xe-2/0/0
xe-2/0/1
xe-2/0/2
xe-2/0/3
xe-2/0/4
xe-2/0/5
xe-2/0/6
xe-2/0/7
xe-1/0/25
xe-1/0/26

1/2.1
1/2.2

VIPRION2TBS

SW-2Port 50

SW2-Port 49

SASN
Cluster # 1

SW1-Port 50

xe-3/0/2
xe-3/0/3
xe-3/0/0
xe-3/0/1

Gx and Gy
Google
Cache

PS3
Te-9/3
Te-9/4
xe-9/0/0
xe-10/0/0
xe-9/1/0
xe-10/1/0

SRX

S R X FW

DNS

DNS

ns1-3g-tbs

ns2-3g-tbs

WAP
GW
Bluecoat

SW1-Port 49

Gi_Switch_TBS2

xe-0/0/35
xe-3/0/6
xe-3/0/7
ge-1/0/30
xe-1/0/2
xe-1/0/1
xe-1/0/0
xe-0/0/31
xe-0/0/30
xe-0/0/26
xe-0/0/25
xe-0/0/21
xe-0/0/20

LAN2
LAN1

Page 8

Force-10

Intersite
Module

Google
Cache

LAN2
LAN1

WAP
GW
WAP
GW

SW2-Port 49

Anti
Anti
Spam
Spam

SW2-Port 50

Gi VAS

SAPC1

xe-3/0/4
xe-3/0/5

xe-2/0/2
xe-2/0/3

xe-1/0/6
xe-1/0/11
xe-1/0/16
xe-1/0/21

SAPC2

SASN
Cluster # 2

xe-2/0/4
xe-2/0/5
xe-2/0/6
xe-2/0/7

Google
Cache

SW1-Port 50

2/2.1
2/2.2
2/2.3
2/2.4

Google
Cache

SW1-Port 49

xe-2/0/0
xe-2/0/1

Google Cache

SW2-Port 49

RD2

1/2.1
1/2.2
1/2.3
1/2.4

Te-4/3

PS1

SASN
Cluster # 3

RD1

xe-0/0/0
xe-0/0/5
xe-0/0/10
xe-0/0/15

Te-4/1
Te-4/2

SW1-Port 49

F5_PBRTBS2

xe-1/0/25
xe-1/0/26

2/2.1
2/2.2
2/2.3
2/2.4

ge-1/0/31
ge-1/0/32
ge-1/0/33
ge-1/0/34

TBS1

SW2-Port 50

RD2

1/2.1
1/2.2
1/2.3
1/2.4

xe-1/0/20
xe-1/0/15
Xe-1/0/10
xe-1/0/5

RD1

xe-1/0/6
xe-1/0/11
xe-1/0/16
xe-1/0/21

IPS
M8000-

SW1-Port 50

F5_PBR_
TBS1

1/2.1
1/2.2

xe-1/0/2
xe-3/0/2
xe-3/0/3
Gi_Switch_TBS1 xe-0/0/20
xe-0/0/21
xe-0/0/25
xe-0/0/26
xe-0/0/30
xe-0/0/31
xe-1/0/0
xe-1/0/1
xe-3/0/4
xe-3/0/5
xe-0/0/35
xe-3/0/6
xe-3/0/7
ge-1/0/30
ge-1/0/31
ge-1/0/32
ge-1/0/33
ge-1/0/34

xe-1/0/20
xe-1/0/15
xe-1/010
xe-1/0/5

F5 PB100

5B
6B
2B
3B

xe-3/0/0
xe-3/0/1

DMZ

Service Topology APN internet/telkomsel

Spam
DNS Gi
Filter
SGSN

gate-router

F5
Load
Balancer

Gn
Network
GGSN
SGSN

Page 9

F5
Load
Balancer

SASN

Internet

gate-router
Proxy

Service Topology APN blackberry.net

SGSN
Blackberry
network

grx-router

VPN Links
via TELIN/AICENT

Gn
Network
GGSN
SGSN

Page 10

grx-router

Blackberry
server

Service Topology APN mms

SGSN

F5
Load
Balancer

Gn
Network
GGSN
SGSN

Page 11

MMS
Proxy

MMSC

Service Topology Roaming APN

Roaming Partner
network

GPRS Roaming Exchange

network

Telkomsel
network

SGSN

/
t.r
ne el.r
r
te s
in om
Gi elk
t

grx-router
via AICENT/TELIN

GRX
Network

Gn
Network

Gi blackberry.net.r

via AICENT/TELIN

grx-router
SGSN

Page 12

GGSN

Gi
m
m
s.r

GGSN Monitoring (1 of 3)

Tools:
Eniq
Cacti http://10.251.15.85/cacti/

Parameters to be monitored:
APN level

Active PDP number

Available IP pool

Uplink & downlink bytes/packets

PDP activation attempt

Node level

Gn, Gi throughput

Routing Engine CPU utilization & temperature

GTP-U & GTP-C CPU utilization & memory usage

Page 13

GGSN Monitoring (2 of 3)

APN level

Page 14

GGSN Monitoring (3 of 3)

Node level

Page 15

First Layer Handling Daily Checklist (1 of 4)

@GGTBS1-re1> show system uptime

Current time: 2013-12-08 21:51:58 WIT
System booted: 2013-12-05 23:45:34 WIT (2d 22:06 ago)
Protocols started: 2013-12-06 00:01:42 WIT (2d 21:50 ago)
Last configured: 2013-12-06 20:57:01 WIT (2d 00:54 ago) by bennypn
9:51PM up 2 days, 22:06, 1 user, load averages: 0.50, 0.42, 0.42

@GGTBS1-re1> show system commit

0
2013-12-06 20:57:01 WIT by bennypn via cli commit synchronize
1
2013-12-06 20:43:16 WIT by bennypn via cli commit synchronize
2
2013-12-06 00:32:52 WIT by ericsson via cli commit synchronize

@GGTBS1-re1> show chassis environment

Class Item
Status
Temp PEM 0
OK
PEM 1
OK
Routing Engine 0
OK
Routing Engine 0 CPU
OK
Routing Engine 1
OK
Routing Engine 1 CPU
OK

Page 16

Pengecekan uptime

Pengecekan perubahan
konfigurasi terakhir

Measurement

43
44
42
45

degrees
degrees
degrees
degrees

C
C
C
C

/
/
/
/

109
111
107
113

degrees
degrees
degrees
degrees

F
F
F
F

Pengecekan
temperature

First Layer Handling Daily Checklist (2 of 4)

@GGTBS1-re1> show chassis routing-engine

Routing Engine status:
Slot 1:
Current state
Master
Election priority
Backup
Temperature
42 degrees C / 107 degrees F
CPU temperature
45 degrees C / 113 degrees F
DRAM
3584 MB
Memory utilization
20 percent
CPU utilization:
User
11 percent
Background
0 percent
Kernel
9 percent
Interrupt
2 percent
Idle
78 percent
Model
RE-A-2000
Serial ID
9009050302
Start time
2013-12-05 23:45:34 WIT
Uptime
2 days, 22 hours, 8 minutes, 20 seconds
Last reboot reason
Router rebooted after a normal shutdown.
Load averages:
1 minute
5 minute 15 minute
0.23
0.36
0.40

Page 17

Pengecekan
routing engine

First Layer Handling Daily Checklist (3 of 4)

@GGTBS1-re1> show ospf neighbor instance Gn_vrf

Address
Interface
State
221.132.243.113 xe-0/0/0.32
Full
221.132.243.121 xe-1/0/0.33
Full

ID
221.132.254.65
221.132.254.73

Pri
1
1

Dead
37
30

@GGTBS1-re1> show ospf neighbor instance Gi_?

Gi_blackberry.r
Gi_inetcorporate
Gi_inetspec
Gi_mms
Gi_telkomsel
Gi_telkomsel.r
Gi_test
Gi_tsel
@GGTBS1-re1> show ospf neighbor instance Gi_telkomsel
Address
Interface
State
ID
192.168.0.10
xe-0/0/0.101
Full
192.168.200.101
192.168.0.11
xe-0/0/0.101
Full
192.168.200.102
192.168.0.26
xe-1/0/0.102
Full
192.168.200.101
192.168.0.27
xe-1/0/0.102
Full
192.168.200.102

Page 18

Pengecekan
routing OSPF
Gn network

Pengecekan
routing OSPF
Gi network

Pri
0
0
0
0

Dead
37
33
31
34

First Layer Handling Daily Checklist (4 of 4)

@GGTBS1-re1> show log messages | last 5

Dec 8 22:07:29 2013 GGTBS1-re1 (FPC Slot 5, PIC Slot 0) gc50
gtpcd[257]:gc-5/0/0:
<PDP>: <04>: pdp_RadiusRecvAcctRespRadius
failure: status=RADIUS_SERVERDOWN, sending gtp failure cause=204
imsi=510100342629156, msisdn=6285203629156, nsapi=5, apn=telkomsel,
idx=0, s2a_wlan=N, type=21, ip=182.12.227.114, ip_dual=-,
sgsn_ip_c=221.132.235.21, sgsn_ip_u=221.132.235.21, teid_data=153047,
pdp_id=aaaaaaaaa2641030 STATE:[SSM = 'CREATE', S =
'PDP_ST_RADIUS_START'], first occurrence at 22:07
Dec 8 22:07:29 2013 GGTBS1-re1 (FPC Slot 4, PIC Slot 0) gc40
gtpcd[253]:gc-4/0/0:
<PDP>: <00>: pdp_DeletePdpRsp: delete rsp from
221.132.233.99 for PDP ID aaaaaaaa80542270 was REJECTED (v1Cause=192),
first occurrence at 22:07
@GGTBS1-re1> show log chassisd | last 5
Dec 6 00:32:49 ch_fru_power_sequencer FEB 3 step
Dec 6 00:32:49 ch_fru_power_sequencer FEB 4 step
Dec 6 00:32:49 ch_fru_power_sequencer FEB 5 step
Dec 6 00:32:49 ... power sequencer finished ...
Dec 6 00:32:49 CHASSISD_PARSE_COMPLETE: Using new

Page 19

0
0
0
configuration

Pengecekan log

First Layer Handling Service Checklist

@GGTBS1-re1> show services epg pgw statistics apn telkomsel
APN: telkomsel
Active PDP contexts: 445128
With traffic class conversational: 0
With traffic class streaming: 0
With traffic class interactive: 444573
With traffic class background: 555
Active EPS bearers: 0
Active dedicated EPS bearers: 0
Active EPS bearers for WLAN: 0
Active IPv6 PDP contexts: 0
Active IPv4v6 PDP contexts: 0
Active IPv6 EPS bearers: 0
Active IPv4v6 EPS bearers: 0
Attempted PDP activations: 65203433, Completed: 64740019

--lines skipped--

Page 20

Radius statistics:
Failed RADIUS Authentication procedures: 0
Failed RADIUS Accounting procedures: 689757
RADIUS accounting servers:
Server IP address: 192.168.0.14
Accounting requests sent: 129384101
Accounting responses received: 128694343
Accounting requests timed out: 689757
Accounting requests retransmitted: 0
Invalid authenticators received: 1

Pengecekan statistik
per APN

First Layer Handling Service Checklist

@GGTBS1-re1> show services epg pgw statistics imsi 510101772143945

International mobile subscriber identity: 510101772143945
Mobile station ISDN number: 6282217143945
NSAPI/Bearer ID: 5
APN: telkomsel
PDP address: 182.12.174.62
Framed Route: 0.0.0.0/ 0
Charging ID: 3779425136
Gi/SGi DiffServ class: AF22, Gn/S5 DiffServ class: AF22
Maximum Bit Rate Downlink (kbps): 8640
Maximum Bit Rate Uplink (kbps): 5760
GTP-C version: 1
Session activation timestamp: Sun Dec 8 22:13:36 2013
Total session duration: 0h 0m 23s
Idle time: 0h 0m 1s
Packets uplink: 166
Packets downlink: 149
Bytes uplink: 37948
Bytes downlink: 61210
Serving Node:
Serving Node Type: SGSN
Control plane IP address: 221.132.227.35
Data plane IP address: 221.132.227.35
Page 21

Pengecekan statistik
user

Escalation Process

1st level - Packet Switch Network Operation Support (PSNOS) team

Moch SN Tsalasanto (Soleh), moch_sn_tsalasanto@telkomsel.co.id,
08111987745
Doddy Widanto, doddy_widanto@telkomsel.co.id, 08119935594

Alex Atmana, alex_atmana@telkomsel.co.id, 08119953112

Benny P Nugraha, benny_p_nugraha@telkomsel.co.id, 08119783464
Dwi SG Utomo, dwi_sg_utomo@telkomsel.co.id, 0811276002

2nd level - Senior Engineer Packet Switch Operation Expert Job

Coordinator
Eko Prasetyo, eko_prasetyo@telkomsel.co.id, 0811939961

Page 22

Question

Page 23

Page 24