Institute for Process Control and Robotics (IPR) URL: http://wwwipr.ira.uka.de
1. Introduction
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
1 Copyright by IPR, 2007
Characteristic components of an autonomous medical robot
Common definitions of robots An industrial robot is an automatically controlled, freely programmable moving mechanism with three or more axes. Its purpose is to grasp and/or move objects for industrial production of goods. A service robot is a freely programmable moving mechanism, which performs semi or fully automated service tasks. An autonomous medical robot is an automatically controlled freely programmable moving mechanism. Its purpose is to handle tools / grippers in the medical field (operating theatre) with the goal of performing diagnosis and therapy at the human being. A telemanipulated medical robot is a robot (usually with several arms), which is controlled by the surgeon in “tele mode”. Its purpose is to handle tools / grippers in the medical field (operating theatre) with the goal of performing diagnosis and therapy at the human being. The semi autonomous mode of robots represents a modus, in which a robot is controlled directly via contact with the operator using automatic control technical methods.
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
3 Copyright by IPR, 2007
2 Copyright by IPR, 2007
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
Characteristic components of robots Mechanics, Kinematics
Mechanics
Drive configuration Measurement system
Programming devices
Materials
Control unit
Control Control unit RC-Control Drive unit Teach pendant Interfaces Software Operating system Robot functions
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
4 Copyright by IPR, 2007
1
Information flow of an autonomous Robot Control (RC)
Pose and frame Frame represents poses and TCP´s of an object with 6 dof. Frame = 4x4 matrix = Relation between two coordinate systems = pose of a coordinate system (B), relative to another coordinate system (A)
Operating, Robot programming User I/O Level
Robot program
Modes of Operation
Path planning and Interpolation xi, yi, zi, ψi, ϕi, ϑi Information processing Level
Interpreter (Compiler)
YB
xi, yi, zi, ψi, ϕi, ϑi
Sensor data processing
Logic data processing Technology control
B
Drive control
Drives
Process I/O Level
Actuators
Sensors
Robot mechanics
5 Copyright by IPR, 2007
4
T5
5
T6
Muff
T4
Muff
3 2 zB
T2
1
Basis
T1
Basis yB Basis
World
T Muff World
TTCP World
xB
T Basis
TTCP = World TBasis ⋅ Basis TTCP
TObject
World
6 Copyright by IPR, 2007
TObject
assistant
TBasis = World TTCP ⋅
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
P = BFA A P
yw
World
TTCP
B
arm 2 camera/ lighting
xw
TTCP = World TBasis ⋅ Basis T1 ⋅1 T2 ⋅ 2 T3 ⋅3 T4 ⋅ 4 T5 ⋅5 T6 ⋅ Muff TTCP Basis
αY YˆB
World
World
(
0
OB
arm 1 instrument
Basis
TTCP
zw
T Muff
0
Xˆ B
αX
Xˆ A
y
zTcp
World
OA
BP
Teleoperated medical robot daVinci for MIS
x
TCP
Basis
R3×3
⎛ xi ⎞ ⎜ ⎟ ⎤ ⎜ yi ⎟ P3×1 ⎥⎥ ~ ⎜ z i ⎟ =⎜ ⎟ ⎥ ⎜ ϕi ⎟ ⎥ ⎜ ⎟ 1 ⎦ ⎜ θi ⎟ ⎜ψ ⎟ ⎝ i⎠
YˆA
z
TTCP
yTcp
T3
BP
αZ
Z2
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
Coordinate systems and Frame arithmetic in Robotics Coordinate systems: World, Basis, Muff, TCP, Object
P2
OA
Z1
⎡ ⎢ FA = ˆ ⎢ ⎢ ⎢ ⎣0
AP
Zˆ B
Poses Pi are represented as frames related to the robot base B coordinate system. A pose has 6 components according to 6 dof. Rotation and Translation of poses are performed via a homogenous matrix (frame) multiplication
Peripherals
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
∆sn
P1
XB
Motion control Direct Inverse Kinematics Kinematics Trans. A 1, A 2, A 3, A 4, A 5, A 6
Y1
ZB
P
Zˆ A
Y2
X2
Ca 1000 worldwide
arm 3 instrument
)
−1 7 Copyright by IPR, 2007
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
8 Copyright by IPR, 2007
2
Semi autonomous mode - Force controlled robotic system Kinemedic
Simplified control of a telemanipulator SDC - Servo drive control SDA - Servo drive amplifier
5 drives per Instrument arm 4 drives for Camera arm
SDA
Console
Speed control
SDC
10+4
-
d dt
+
Generation of position command values • Scaling • Smoothing
Position measurement
position encoder per drive
Position control
-
12
+
Manipulator
Position encoder 2 Input devices with 5dof each 3T, 2R, 1Gripper Control, 2 Modi Arm control: 10dof Instrument control: 4dof 9 Copyright by IPR, 2007
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
10 Copyright by IPR, 2007
Semi autonomous mode – Force controlled robot control Force-Torque-Sensor
-
F
Position F
-
P
Interpolator
Force/Contact (force&localization) Reflex signal
-
F
-
P
Interpolator +
C
-
Contact
P
-
Interpolator
-
Interpolator
-
I. Kin.
Control
Roboter
2. Robot assisted Surgery with program controlled autonomous robots
Reflex selector P soll
Artificial Skin University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
11 Copyright by IPR, 2007
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
12 Copyright by IPR, 2007
3
Autonomous medical robot Cyberknife
Research: Motion compensation
movement of thorax
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
13 Copyright by IPR, 2007
prediction model
correlation model
movement of tumor
14 Copyright by IPR, 2007
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
Computer- and Robot- Aided Head Surgery-Workflow (IPR)
Autonomous Robots for Osteotomy (IPR)
Zo
OSC Yo
Xo P3 P2 P1 P0
P4
ks
ks
P5
P6
Simulation
P7
P8
Operation Planning
Registration
RobaCKa University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
Image Acquisition 15 Copyright by IPR, 2007
Validation
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
Augmented Reality
Surgical Robotics 16 Copyright by IPR, 2007
4
Transfer of the planned Trajectory to a robot program (IPR)
Registration (IPR)
The robot points are represented relative to an object coordinate system, which is located at the head of the patient OCS (Object- CS)
HOME
PTP HOME ;Home position
Zocs
;P0={x- 100, y 100, z 0, a 0, b 90, c 0} ;Start position on bone surface
LIN P1
;P1={x- 100, y 100, z- 10, a 0, b 90, c 0}
Yocs Xocs P3
P5
P4
O (Object-
HOME
PTP P0
P6
P1 P0 P7
LIN P2
;P2={x- 120, y 120, z- 10, a 0, b 90, c 0}
LIN P3
;P3={x- 150, y 155, z- 10, a 0, b 90, c 0}
W
TO
Yocs
RRB
Xocs P3
;Move into bone z- 10
P2
CS)
Zocs
W = World CS = Coordinate System B = Basis Robot CS RRB = Robot Rigid Body CS
P4
P5
TO
Zw
W
P2
RRB
P6
P1
B
P0 P7
:
Zw
z
World -
B
LIN P8
;P8={x- 20, y 125, z- 10, a 0, b 90, c 0}
LIN P1
;Move out z=0
LIN P0
;Back to start position
B
Given W
PTP HOME
x
y CS
Xw
TO
:
Xw
CS
P8
P8
Yw
W Yw
T RRB
Pi
y
B
z
x
T RRB
TRRB , W TRRB
17 Copyright by IPR, 2007
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
Patient registration with markers (IPR)
Wanted: BPi B
Pi = B TO ⋅O Pi
B
TO = B TW ⋅W TO
B
−1 TW = W T RRB ⋅ B T RRB
B
TO by measurement of O
B
RBasis
TW
18 Copyright by IPR, 2007
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
Alternative non-invasive, markerless registration with surface matching (IPR)
Preoperativ marker (4screws) identification: Scan with coded light approach and registration with surface matching:
Monitoring and localization with a navigation system
Intraoperativ marker identification: Scan
Least-squares fitting
CT/MRT
Video projector Tracking with a dental adapter:
With the help of the markers the actual object coordinate system of the patient in the OR is calculated University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
19 Copyright by IPR, 2007
OR prototype
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
20 Copyright by IPR, 2007
5
Assistive Robot System RobaCKa (IPR)
Monitoring to get a high safety (IPR) Exact cutting of trajectories
Central Computer
INS
COM PC
ROB
KMS KMS
GUI
Data/Digital Signal/Emergency Stop
Robot Control
`
General surveillance `
Force Torque Sensor
Surgeon
Overload Protection
pi+1
ds Infrared Navigation System
pi
Force torque sensor feed control monitoring of limits
pi -1 `
dt
Infrared navigation system lateral deviance ds deviance in depth dt
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
21 Copyright by IPR, 2007
Clinical Evaluation of the Surgical Robot (IPR)
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
23 Copyright by IPR, 2007
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
22 Copyright by IPR, 2007
Autonomous Medical Robot for precise drilling for Cochlea Implantation (IPR)
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
24 Copyright by IPR, 2007
6
Advantages of future bone cutting methods for robots (IPR) Milling:
Laser:
Force control Bone meal
No forces No vibrations No necroses No metal attrition Less loss of bone Finer cuts
25 Copyright by IPR, 2007
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
CO2-Laser cutting (IPR)
New Cutting Opportunities with Laser (IPR)
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
26 Copyright by IPR, 2007
Robot with Laser (IPR)
pulsed CO2 laser (80 µs, 80 mJ)
multipass cutting to reduce rest heat accumulation assisting spray
strong focusing cut width ≥ 100 µm
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
27 Copyright by IPR, 2007
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
28 Copyright by IPR, 2007
7
Experiments (IPR)
Repositioning of Scanning Head (IPR)
29 Copyright by IPR, 2007
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
New Vision control method for laser ablation (IPR)
Light source
Scanner
CO2-Laser Beam combiner
CO2 Laser beam
Microscope
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
30 Copyright by IPR, 2007
Visual detection of the membrane for Cochleostomy (IPR) Automatic detection of the areas of break through (red) (fast marching algorithm) Challenge: additional detection of the membrane (green)
Camera
Water spray
towards microscope Illumination
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
Bone
31 Copyright by IPR, 2007
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
32 Copyright by IPR, 2007
8
Orthognathic surgery for the correction of Dentofacial skeletal irregularities with an autonomous robot (IPR)
Conclusion (IPR)
Laser superior to conventional drilling
Only useful with robot guided laser and a beam scanning head
Today: Experience for repositioning the bone segments to create a good outlook of the patient, manual measurements Idea: Assistive robot system for holding the bone segments in the preplanned position during fixation procedure 1. Acquire initial and target position of the maxilla in the articulator Store relative transformation
2. Build individual maxillary splint from light curing resin (leicht aushärtendes Harz) University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
33 Copyright by IPR, 2007
Developed devices for orthognathic surgery (IPR)
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
34 Copyright by IPR, 2007
Robot Assisted Fracture Reduction (TUB/IRP) with an autonomous robot
Torque sensor
Maxillary splint
Tool adapter
Maxilla fixed with the splint University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
35 Copyright by IPR, 2007
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
36 Copyright by IPR, 2007
9
Teleoperated medical robot daVinci for MIS
arm 1 instrument arm 2 camera/ lighting
3. Teleoperated medical robots
assistant
Ca 1000 worldwide
arm 3 instrument
37 Copyright by IPR, 2007
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
University of Karlsruhe, Prof. Dr.-Ing. Heinz Wörn Institute for Process Control and Robotics (IPR)
General research fields: Motion controlled Telemanipulation
Current Research at IPR: port & pose planning Problem: Best poses of the arms and ports Planning
5dof 4dof
Instrument/ Robot modeling Geometry Kinematics
Patient modeling
6dof (passive) 5dof (active)
camera 4dof
Situs (insufflated) Obstacles Critical structures Access area
Movement compensation of organs (beating heart, ...) for interaction and Augmented Reality (Vessels, blood flows...)
Robot tracking control
Sensors
Prediction (q.v. GK 1126)
Haptics
Transfer
Instruments (tracking) (Ø
