Adaptive Parallel Simulation

of Large Finite Element

Problems

Babak Kamrani Boris Jeremic

Computational Geomechanics Group
University of California Davis
July 22, 2013

Outline
Motivation
Parallel elastic plastic finite element ,
challenges and difficulties
Examples
Summary

Outline
Motivation
Challenges in elastic plastic finite
element
Examples
Summary

Motivation
Reliable and precise finite element
simulation of the behavior of complex
earthquake soil foundation and structure
interaction .
Lack of parallel Inelastic finite element
tool.
Utilization of the capabilities of multicore
processors.

Complexities of ESFSI

3D, Inclined seismic waves

( body and surface waves)
Full Coupling of pore fluid
with soil/rock skeleton.
Wide rage of excitation
frequencies ( up to 50 Hz)
Inelastic solids and
structures (material and
components
Uncertainty in material
and load

Multi core CPU

AMD Opteron(TM) Processor 6274

Cluster of multicore CPUs

CORE
Multi Core CPU

Main Board

Inter CPU
Connection
Main Board
Connection
LAN
Connection

Outline
Motivation
Parallel elastic plastic finite element ,
challenges and difficulties
Examples
Summary

Domain Decomposition

Processor 1

Processor 2

Processor93

Elastic plastic finite element

Elastic computation:
=

Elastic plastic computation:

Integration of constitutive equations at
each integration points
Consistency condition
should be satisfied

=0

10

Challenges in parallel Elastic Plastic

FE

Dynamic and unpredictable change of the

elastic plastic zones pattern
Unbalanced computational load in the
processors
Computational load balancing is required
frequently
Data redistribution difficulties
Dual phase computational load balancing:
Element Level
Equation Level
11

Execution of adaptive simulation

Iterative Mesh-based Computation

Mesh Adaptation

Compute a New Partitioning

Redistribute Data

12
12

Graph of FE mesh

FE MESH

MAPPING

GRAPH

13

Graph partitioning and domain

decomposition for FE mesh

Five Edges Cut

Decomposed Mesh

Partitioned Graph
14

Weighed Graph Partitioning

2
1
2

2
2

5
3

Top Partition Weight = 5+5+2

Bottom Partition Weight = 1+3+4

Edge cut Cost = 3+3+1+6+4

6
3
Partitioned Graph

Your logo here

15

Multilevel Graph Partitioning

2 1
2
3

5
3

3
1

6
2

2 1

4
4 1

5
3 3

1
1

5
3

5
3

6
2

5
3 3

1
1

5
6

4
4 1

Partitioning
16

5
2

Balanced partitioning

Elastic

Mild Plasticity

Severe Plasticity

Jeremic et al, 2005

17

Outline
Motivation
Parallel elastic plastic finite element ,
challenges and difficulties
Examples
Summary

18

Elastic Plastic small block of brick

Elements
9
9
9

9
7

11
Number of elements

3X3X3

Number of Dofs:

882

11

Number of the partitions

3
13

19

Elastic Plastic small block of brick

elements
30
25
20
15
10
5
0

Adaptive
domain
decomosition
Static domain
decomposition

20

Elastic Plastic block of brick

elements
1200
1000
800
600
400
200
0

Number of 27 node Bricks

10X10X10

Number of dofs

26460

Adaptive
domain
Decomosition
Static domain
decomposition

21

Elastic Plastic block of brick

elements
1200
1000
800
600
400
200
0

Adaptive domain Decomosition

Static domain decomposition

Number of 27 node bricks 20X50X10

Number of dofs

248460
22

Current model Nuclear Power

Plant

25000 ,27 node brick elements

Number of 27 node bricks

Number of 4 node shells
Number of dofs
23

NPP free field model

Number of 27 Node bricks

Number of dofs

633234

24

NPP free field model

25

Outline
Motivation
Parallel elastic plastic finite element ,
challenges and difficulties
Examples
Summary

26

Summary and current work

Frequency scaling is CPUs are prevented in the

processors because of the physical limitations
Adaptive parallel finite element employing
graph partitioning techniques is efficient and
practical way to simulate the elastic plastic
problem
Simulation should be kept balanced, data
migration cost should be optimized in the
course of the simulation
It is extensible to different cluster architectures
and different type of elements, constitutive
models can be used
27

Questions and Answers

28