Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
EmbeddedLinux
kernelanddriver
development
MichaelOpdenacker
ThomasPetazzoni
FreeElectrons
1
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Rightstocopy
Copyright20042008,FreeElectrons
feedback@freeelectrons.com
Documentsources,updatesandtranslations:
http://freeelectrons.com/docs/kernel
Corrections,suggestions,contributionsand
translationsarewelcome!
AttributionShareAlike3.0
Youarefree
tocopy,distribute,display,andperformthework
tomakederivativeworks
tomakecommercialuseofthework
Underthefollowingconditions
Latestupdate:Jan3,2009
Attribution.Youmustgivetheoriginalauthorcredit.
ShareAlike.Ifyoualter,transform,orbuilduponthiswork,you
maydistributetheresultingworkonlyunderalicenseidenticalto
thisone.
Foranyreuseordistribution,youmustmakecleartoothersthelicense
termsofthiswork.
Anyoftheseconditionscanbewaivedifyougetpermissionfromthe
copyrightholder.
Yourfairuseandotherrightsareinnowayaffectedbytheabove.
Licensetext:http://creativecommons.org/licenses/bysa/3.0/legalcode
2
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Bestviewedwith...
ThisdocumentisbestviewedwitharecentPDFreader
orwithOpenOffice.orgitself!
Takeadvantageofinternalorexternalhyperlinks.
So,donthesitatetoclickonthem!Seenextpage.
Findpagesquicklythankstoautomaticsearch
Usethumbnailstonavigateinthedocumentinaquickway
IfyourereadingapaperorHTMLcopy,youshouldgetyour
copyinPDForOpenOffice.orgformaton
http://freeelectrons.com/training/drivers!
3
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Hyperlinksinthisdocument
Linkstoexternalsites
UsableinthePDFandODPformats
Example:http://kernel.org/ Trythemonthispage!
Kernelsourcefiles
Ourlinksletyouviewtheminyourbrowser.
Example:kernel/sched.c
click
KernelsourcecodeIdentifiers:functions,macros,type
definitions...
Yougetaccesstotheirdefinition,implementationandwherethey
areused.Thisinvitesyoutoexplorethesourcebyyourself!
wait_queue_head_tqueue;
init_waitqueue_head(&queue);
TableofcontentsDirectlyjumptothecorrespondingsections.
Example:Kernelconfiguration
4
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Courseprerequisites
Skillstomaketheselecturesandlabsprofitable
FamiliaritywithUnixconceptsanditscommandlineinterface
Essentialtomanipulatesourcesandfiles
Essentialtounderstandanddebugthesystemthatyoubuild
Youshouldreadhttp://freeelectrons.com/training/intro_unix_linux
ThisUnixcommandlineinterfacetrainingalsoexplainsUnixconcepts
notrepeatedinthisdocument.
ExperiencewithCprogramming
OnlineCcoursescanbefoundon
http://dmoz.org/Computers/Programming/Languages/C/Tutorials/
5
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Contents(1)
Kerneloverview
Linuxfeatures
Kernelcode
Kernelsubsystems
Linuxversioningschemeanddevelopmentprocess
Legalissues
Kerneluserinterface
6
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Contents(2)
Compilingandbooting
Bootloaders
Linuxkernelsources
Linuxdevicefiles
Kernelsourcemanagers
Crosscompilingthekernel
Kernelconfiguration
Basicdriverdevelopment
Compilingthekernel
Loadablekernelmodules
Overallsystemstartup
Moduleparameters
Addingsourcestothetree
7
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Contents(3)
Driverdevelopment
Memorymanagement
Sleeping,Interruptmanagement
I/Omemoryandports
Handlingconcurrency
Characterdrivers
mmap
Debugging
Devicemodel,sysfs
Processesandscheduling
8
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Contents(4)
Adviceandresources
Annexes
Gettinghelpandcontributions
Quizanswers
Bugreportandpatchsubmission
Kernelsources
References
Slabcachesandmemorypools
Lastadvice
Initrunlevels
9
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Kerneloverview
Linuxfeatures
10
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Studiedkernelversion:2.6
Linux2.6
Linux2.6.0wasreleasedinDecember2003.
Lotsoffeaturesandnewdrivers
havebeenaddedataquickpacesincethen.
Itisgettingmoreandmoredifficulttogetsupportordrivers
forrecenthardwarein2.4.Nocommunitysupportatall!
Thesetrainingslidesshouldbecompliantwithatleast
Linux2.6.26.Wealsomentionfeaturesinmorerecent
kernels.
11
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Linuxkernelkeyfeatures
Portabilityandhardware
support
Runsonmostarchitectures.
Security
Itcan'thideitsflaws.Itscode
isreviewedbymanyexperts.
Scalability
Canrunonsupercomputers
aswellasontinydevices
(4MBofRAMisenough).
Stabilityandreliability.
Compliancetostandardsand
interoperability.
Exhaustivenetworking
support.
Modularity
Canincludeonlywhata
systemneedsevenatrun
time.
Easytoprogram
Youcanlearnfromexisting
code.Manyusefulresources
onthenet.
12
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Supportedhardwarearchitectures
Seethearch/directoryinthekernelsources
Minimum:32bitprocessors,withorwithoutMMU
32bitarchitectures(arch/subdirectories)
arm,avr32,cris,frv,h8300,i386,m32r,m68k,
m68knommu,mips,parisc(canhavea64bitkernelbutalways
a32bituserspace),powerpc,ppc,s390,sh,sparc,um,v850,
xtensa
64bitarchitectures:
alpha,ia64,mips,powerpc,sh64,sparc64,x86_64
Seearch/<arch>/Kconfig,arch/<arch>/README,or
Documentation/<arch>/fordetails
13
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Kerneloverview
Kernelcode
14
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
ImplementedinC
ImplementedinClikeallUnixsystems.
(CwascreatedtoimplementthefirstUnixsystems)
AlittleAssemblyisusedtoo:
CPUandmachineinitialization,exceptions,
andcriticallibraryroutines.
Seehttp://www.tux.org/lkml/#s153
forreasonsfornotusingC++
(mainreason:thekernelrequiresefficientcode).
15
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
CompiledwithGNUC
NeedGNUCextensionstocompilethekernel.
So,youcannotuseanyANSICcompiler!
YoucanalsousetheIntelandMarvellcompilers(onlyontheir
respectiveplatforms)whichidentifythemselvesasaGNUcompiler.
SomeGNUCextensionsusedinthekernel:
InlineCfunctions
Inlineassembly
Structurememberinitializationinanyorder(alsoinANSIC99)
Branchannotation(seenextpage)
Requiresatleastgcc3.2.
SeeDocumentation/Changesinkernelsources.
16
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Helpgcctooptimizeyourcode!
Usethelikelyandunlikelystatements
(include/linux/compiler.h)
Example:
if(unlikely(err)){
...
}
TheGNUCcompilerwillmakeyourcodefaster
forthemostlikelycase.
Usedinmanyplacesinkernelcode!
Don'tforgettousethesestatements!
17
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
NoClibrary
Thekernelhastobestandaloneandcan'tuseuserspace
code.
Userspaceisimplementedontopofkernelservices,notthe
opposite.
Kernelcodehastosupplyitsownlibraryimplementations
(stringutilities,cryptography,uncompression...)
So,youcan'tusestandardClibraryfunctionsinkernelcode.
(printf(),memset(),malloc()...).
YoucanalsousekernelCheaders.
Fortunately,thekernelprovidessimilarCfunctionsforyour
convenience,likeprintk(),memset(),kmalloc()...
18
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Managingendianism
Linuxsupportsbothlittleandbigendianarchitectures
Eacharchitecturedefines__BIG_ENDIANor
__LITTLE_ENDIANin<asm/byteorder.h>
Canbeconfiguredinsomeplatformssupportingboth.
Tomakeyourcodeportable,thekerneloffersconversion
macros(thatdonothingwhennoconversionisneeded).Most
usefulones:
u32cpu_to_be32(u32); //CPUbyteordertobigendian
u32cpu_to_le32(u32); //CPUbyteordertolittleendian
u32be32_to_cpu(u32); //BigendiantoCPUbyteorder
u32le32_to_cpu(u32); //LittleendiantoCPUbyteorder
19
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Kernelcodingguidelines
Neverusefloatingpointnumbersinkernelcode.Your
codemayberunonaprocessorwithoutafloatingpoint
unit(likeonarm).Floatingpointcanbeemulatedbythe
kernel,butthisisveryslow.
Defineallsymbolsasstatic,exceptexportedones
(toavoidnamespacepollution)
SeeDocumentation/CodingStyleformore
guidelines
20
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Kerneloverview
Kernelsubsystems
21
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Kernelarchitecture
App1
App2
...
User
space
Clibrary
Systemcallinterface
Process
management
Memory
management
Filesystem
support
Device
control
Networking
Character
devicedrivers
Network
devicedrivers
Filesystem
types
CPUsupport
code
CPU/MMU
supportcode
Storage
drivers
Kernel
space
Hardware
CPU
RAM
Storage
22
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Kernelmemoryconstraints
Whocanlookafterthekernel?
User
process
Nomemoryprotection
Accessingillegalmemory
locationsresultin(oftenfatal)
kerneloopses.
Attempt
toaccess
Fixedsizestack(8or4KB)
Unlikeinuserspace,
nowaytomakeitgrow.
Illegal
memory
location
Kernelmemorycan'tbe
swappedout(forthesame
reasons).
SIGSEGV,kill
Kernel
Exception
(MMU)
Userspacememory
management
Usedtoimplement:
memoryprotection
stackgrowth
memoryswappingtodisk
demandpaging
23
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
I/Oschedulers
MissionofI/Oschedulers:reorderreadsandwritestodisk
tominimizediskheadmoves(timeconsuming!)
Slower
Faster
Notneededinembeddedsystemswithnoharddisks
(dataaccesstimeindependentoflocationonflashstorage)
BuildyourkernelwithnoopI/Oschedulerthen!
24
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Kerneloverview
Linuxversioningschemeanddevelopmentprocess
25
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Until2.6(1)
Onestablemajorbranchevery2or3years
Identifiedbyanevenmiddlenumber
Examples:1.0,2.0,2.2,2.4
Onedevelopmentbranchtointegratenewfunctionalitiesand
majorchanges
Identifiedbyanoddmiddlenumber
Examples:2.1,2.3,2.5
Aftersometime,adevelopmentversionbecomesthenew
baseversionforthestablebranch
Minorreleasesonceinwhile:2.2.23,2.5.12,etc.
26
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Until2.6(2)
Stableversion
2.4.0
2.4.1
2.4.2
2.4.3
2.4.4
2.5.0
2.4.5
2.5.1 2.5.2
2.4.6
2.5.3
Development
2.4.7
2.5.4
2.6.0
2.4.8
2.6.1
Stable
Note:inreality,manymoreminor
versionsexistinsidethestableand
developmentbranches
27
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
ChangessinceLinux2.6(1)
Since2.6.0,kerneldevelopershavebeenableto
introducelotsofnewfeaturesonebyoneonasteadypace,
withouthavingtomakemajorchangesinexisting
subsystems.
OpeninganewLinux2.7(or2.9)developmentbranchwill
berequiredonlywhenLinux2.6isnolongerableto
accommodatekeyfeatureswithoutundergoingtraumatic
changes.
Thankstothis,morefeaturesarereleasedtousersata
fasterpace.
28
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
ChangessinceLinux2.6(2)
Since2.6.14,thekerneldevelopersagreedonthefollowingdevelopment
model
Afterthereleaseofa2.6.xversion,atwoweeksmergewindow
opens,duringwhichmajoradditionsaremerged
Themergewindowisclosed
bythereleaseoftestversion2.6.(x+1)rc1
Thebugfixingperiodopens,forsixtotenweeks
Atregularintervalsduringthebugfixingperiod,
2.6.(x+1)rcYtestversionsarereleased
Whenconsideredsufficientlystable,
kernel2.6.(x+1)isreleased,andtheprocessstartsagain
29
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Mergeandbugfixingwindows
2weeks
6to10weeks
Mergewindow
2.6.21
Bugfixingperiod
2.6.22rc1
2.6.22rc2
2.6.22rc3
2.6.22rc5
2.6.22rc4
2.6.22
30
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Morestabilityforthe2.6kerneltree
Issue:securityfixesonlyreleasedforlast(orlasttwo)stable
kernelversions(like2.6.16and2.6.17),andofcourseby
distributionsfortheexactversionthatyou'reusing.
Somepeopleneedtohavearecentkernel,butwithlong
termsupportforsecurityupdates.
That'swhyAdrianBunkproposedtomaintaina2.6.16
stabletree(March2006),foraslongasneeded(years!).
StillsupportedinJuly2008(2.6.16.62).
31
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
NostableLinuxinternalAPI(1)
Ofcourse,theexternalAPImustnotchange(systemcalls,/proc,/sys),
asitcouldbreakexistingprograms.Newfeaturescanbeadded,butkernel
developerstrytokeepbackwardcompatibilitywithearlierversions,atleast
for1orseveralyears.
TheinternalkernelAPIcannowundergochangesbetweentwo2.6.x
releases.Astandalonedrivercompiledforagivenversionmaynolonger
compileorworkonamorerecentone.
SeeDocumentation/stable_api_nonsense.txt
inkernelsourcesforreasonswhy.
WheneveradeveloperchangesaninternalAPI,(s)healsohastoupdateall
kernelcodewhichusesit.Nothingbroken!
Worksgreatforcodeinthemainlinekerneltree.
Difficulttokeepinlineforoutoftreeorclosedsourcedrivers!
32
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
NostableLinuxinternalAPI(2)
USBexample
LinuxhasupdateditsUSBinternalAPIatleast3times(fixes,
securityissues,supportforhighspeeddevices)andhasnowthe
fastestUSBbusspeeds(comparedtoothersystems)
WindowsXPalsohadtorewriteitsUSBstack3times.But,
becauseofclosedsource,binarydriversthatcan'tbeupdated,they
hadtokeepbackwardcompatibilitywithallearlierimplementation.
Thisisverycostly(development,security,stability,performance).
SeeMyths,Lies,andTruthsabouttheLinuxKernel,byGregK.H.,
fordetailsaboutthekerneldevelopmentprocess:
http://kroah.com/log/linux/ols_2006_keynote.html
33
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
What'snewineachLinuxrelease?(1)
commit3c92c2ba33cd7d666c5f83cc32aa590e794e91b0
Author:AndiKleen<ak@suse.de>
Date:TueOct1101:28:332005+0200
[PATCH]i386:Don'tdiscardupper32bitsofHWCRonK8
Needtouselonglong,notlongwhenRMWingaMSR.Ithink
it'sharmlessrightnow,butstillshouldbebetterfixed
ifAMDaddsanybitsintheupper32bitofHWCR.
BugwasintroducedwiththeTLBflushfilterfixfori386
Signedoffby:AndiKleen<ak@suse.de>
Signedoffby:LinusTorvalds<torvalds@osdl.org>
...
?? !
TheofficiallistofchangesforeachLinuxreleaseisjusta
hugelistofindividualpatches!
Verydifficulttofindoutthekeychangesandtogetthe
globalpictureoutofindividualchanges.
34
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
What'snewineachLinuxrelease?(2)
Fortunately,asummaryofkeychanges
withenoughdetailsisavailableon
http://wiki.kernelnewbies.org/LinuxChanges
?? !
Foreachnewkernelrelease,youcanalsogetthe
changesinthekernelinternalAPI:
http://lwn.net/Articles/2.6kernelapi/
What'snext?
Documentation/featureremovalschedule.txt
liststhefeatures,subsystemsandAPIsthatare
plannedforremoval(announced1yearinadvance).
35
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Kerneloverview
Legalissues
36
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Linuxlicense
ThewholeLinuxsourcesareFreeSoftwarereleased
undertheGNUGeneralPublicLicenseversion2(GPLv2).
Seeourhttp://freeelectrons.com/articles/freesw/training
fordetailsaboutFreeSoftwareanditslicenses.
37
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Linuxkernellicensingconstraints
Constraintsatreleasetime(noconstraintbefore!)
ForanydeviceembeddingLinuxandFreeSoftwarelicensedundera
copyleftlicense,youhavetoreleasesourcestotheenduser.Youhave
noobligationtoreleasethemtoanybodyelse!
AccordingtotheGPL,allderivedworksofaworkmustbereleasedunder
thesamelicense,theGPL
GreyareaaroundLinuxdrivers:
aretheyderivedworksofthekernelornot?
Proprietarydriversarelessandlesstolerated,
http://www.linuxfoundation.org/en/Kernel_Driver_Statement
Proprietarydriversmustnotbestaticallycompiledintothekernel.
Youarenotallowedtoreusecodefromotherkerneldrivers(GPL)
inaproprietarydriver.
38
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
AdvantagesofGPLdrivers
Fromthedriverdeveloper/decisionmakerpointofview
Youdon'thavetowriteyourdriver
fromscratch.Youcanreusecode
fromsimilarfreesoftwaredrivers.
Yougetfreecommunity
contributions,support,codereview
andtesting.Proprietarydrivers
(evenwithsources)don'tgetany.
Yourdriverscanbefreelyshipped
byothers(mainlybydistributions).
Closedsourcedriversoftensupport
agivenkernelversion.Asystem
withclosedsourcedriversfrom2
differentsourcesisunmanageable.
Usersandthecommunitygeta
positiveimageofyourcompany.
Makesiteasiertohiretalented
developers.
Youdon'thavetosupplybinary
driverreleasesforeachkernel
versionandpatchversion(closed
sourcedrivers).
Drivershaveallprivileges.You
needthesourcestomakesurethat
adriverisnotasecurityrisk.
Yourdriverscanbestatically
compiledintothekernel.
39
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Advantagesofintreekerneldrivers
Advantagesofhavingyourdriversinthemainlinekernelsources
Onceyoursourcesareacceptedinthemainlinetree,theyare
maintainedbypeoplemakingchanges.
Costfreemaintenance,securityfixesandimprovements.
Easyaccesstoyoursourcesbyusers.
Manymorepeoplereviewingyourcode.
40
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
LegalproprietaryLinuxdrivers(1)
Linuxkernel
Wrapper
SpecialAPI
WorkingaroundtheGPLbycreatingaGPLwrapper:
Binary
blob
(proprietary
driver)
Theproprietaryblobisnotbrokenwhenyourecompileorupdate
thekerneland/ordriver.Hence,theproprietarydrivermaynotbe
consideredasaderivativework.However,thekernelismonolithic
andtheblobstillbelongstoasingleexecutable.Thisisstill
controversial!
41
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
LegalproprietaryLinuxdrivers(2)
2examplecases
Nvidiagraphiccarddrivers
Supportingwirelessnetworkcards
usingWindowsdrivers.
TheNdisWrapperproject
(http://ndiswrapper.sourceforge.net/)
implementstheWindowskernelAPI
andNDIS(NetworkDriverInterface
Spec.)APIwithintheLinuxkernel.
Usefulforusingcardsforwhichno
specificationsarereleased.
Drawbacks
Stillsomemaintenanceissues.
Example:Nvidiaproprietarydriver
incompatiblewithX.org7.1.
Performanceissues.
Wrapperoverheadandoptimizations
notavailable.
Securityissues.Thedriversare
executedwithfullkernelprivileges.
...andallotherissueswith
proprietarydrivers.Userslosemost
benefitsofFreeSoftware.
42
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Kerneloverview
Kerneluserinterface
43
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Mountingvirtualfilesystems
Linuxmakessystemandkernelinformationavailablein
userspacethroughvirtualfilesystems(virtualfilesnot
existingonanyrealstorage).Noneedtoknowkernel
programmingtoaccessthis!
Mounting/proc:
sudomounttprocnone/proc
Mounting/sys:
sudomounttsysfsnone/sys
Filesystemtype
Mountpoint
Rawdevice
orfilesystemimage
Inthecaseofvirtual
filesystems,anystringisfine
44
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Kerneluserspaceinterface
Afewexamples:
/proc/cpuinfo:processorinformation
/proc/meminfo:memorystatus
/proc/version:kernelversionandbuildinformation
/proc/cmdline:kernelcommandline
/proc/<pid>/environ:callingenvironment
/proc/<pid>/cmdline:processcommandline
...andmanymore!Seebyyourself!
45
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Userspaceinterfacedocumentation
Lotsofdetailsaboutthe/procinterfaceareavailablein
Documentation/filesystems/proc.txt
(almost2000lines)inthekernelsources.
Youcanalsofindotherdetailsintheprocmanualpage:
manproc
SeetheNewDeviceModelsectionfordetailsabout/sys
46
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Userspacedevicedrivers(1)
Possibletoimplementdevicedriversinuserspace!
Suchdriversjustneedaccesstothedevicesthrough
minimum,generickerneldrivers.
Examples
Printerandscannerdrivers
(ontopofgenericparallelport/USBdrivers)
Xdrivers:lowlevelkerneldrivers+userspaceXdrivers.
UserspacedriversbasedonUIO,frameworkintroducedin
2.6.23
47
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Userspacedevicedrivers(2)
Advantages
Noneedforkernelcodingskills.Easiertoreusecodebetween
devices.
Driverscanbewritteninanylanguage,evenPerl!
Driverscanbekeptproprietary.
Drivercodecanbekilledanddebugged.Cannotcrashthekernel.
Canbeswappedout(kernelcodecannotbe).
Canusefloatingpointcomputation.
Lessinkernelcomplexity.
Drawbacks
Lessstraightforwardtohandleinterrupts.
Increasedlatencyvs.kernelcode.
SeeUIOHowtointhekerneldocumentationfordetailedinformations
andtheUsingUIOonanEmbeddedplatformtalkatELC2008,
http://www.celinux.org/elc08_presentations/uio080417celfelc08.pdf.
48
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
CompilingandbootingLinux
Linuxkernelsources
49
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
kernel.org
Downloadkernel
sourcesfrom
http://kernel.org
50
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Linuxsourcesstructure(1)
arch/<arch>
arch/<arch>/<mach>
block/
COPYING
CREDITS
crypto/
Documentation/
drivers/
fs/
include/
include/asm<arch>
include/linux
init/
Architecturespecificcode
Machine/boardspecificcode
Blocklayercore
Linuxcopyingconditions(GNUGPL)
Linuxmaincontributors
Cryptographiclibraries
Kerneldocumentation.Don'tmissit!
Alldevicedriversexceptsoundones(usb,pci...)
Filesystems(fs/ext3/,etc.)
Kernelheaders
Architectureandmachinedependentheaders
Linuxkernelcoreheaders
Linuxinitialization(includingmain.c)
51
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Linuxsourcesstructure(2)
ipc/
Kbuild
kernel/
lib/
MAINTAINERS
Makefile
mm/
net/
README
REPORTINGBUGS
samples/
scripts/
security/
sound/
usr/
Codeusedforprocesscommunication
Partofthekernelbuildsystem
Linuxkernelcore(verysmall!)
Misclibraryroutines(zlib,crc32...)
Maintainersofeachkernelpart.Veryuseful!
TopLinuxmakefile(setsarchandversion)
Memorymanagementcode(smalltoo!)
Networksupportcode(notdrivers)
Overviewandbuildinginstructions
Bugreportinstructions
Samplecode(markers,kprobes,kobjects)
Scriptsforinternalorexternaluse
Securitymodelimplementations(SELinux...)
Soundsupportcodeanddrivers
Codetogenerateaninitramfscpioarchive.
52
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Linuxkernelsize(1)
Linux2.6.26sources:
Rawsize:264MB(25700files,approx8.5millionlinesofcode)
gzipcompressedtararchive:60MB
bzip2compressedtararchive:48MB(better)
lzmacompressedtararchive:39MB(best)
MinimumcompiledLinuxkernelsize(withLinuxTinypatches)
approx300KB(compressed),800KB(raw)(Linux2.6.14)
Whyarethesesourcessobig?
Becausetheyincludethousandsofdevicedrivers,manynetwork
protocols,supportmanyarchitecturesandfilesystems...
TheLinuxcore(scheduler,memorymanagement...)isprettysmall!
53
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Linuxkernelsize(2)
SizeofLinuxsourcedirectories(KB)
arch
block
crypto
Documentation
drivers
fs
include
init
ipc
kernel
lib
mm
net
scripts
security
sound
usr
Linux2.6.17
Measuredwith:
dusapparentsize
0
50000
100000
150000
54
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
GettingLinuxsources:2possibilities
Fullsources
Theeasiestway,butlongertodownload.
Example:
http://kernel.org/pub/linux/kernel/v2.6/linux2.6.14.7.tar.bz2
Orpatchagainstthepreviousversion
Assumingyoualreadyhavethefullsourcesofthepreviousversion
Example:
http://kernel.org/pub/linux/kernel/v2.6/patch2.6.14.bz2(2.6.13to2.6.14)
http://kernel.org/pub/linux/kernel/v2.6/patch2.6.14.7.bz2(2.6.14to2.6.14.7)
55
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Downloadingfullkernelsources
Downloadingfromthecommandline
Withawebbrowser,identifytheversionyouneedon
http://kernel.org
Intherightdirectory,downloadthesourcearchiveanditssignature
(copyingthedownloadaddressfromthebrowser):
wgethttp://kernel.org/pub/linux/kernel/v2.6/linux2.6.11.12.tar.bz2
Extractthecontentsofthesourcearchive:
tarjxflinux2.6.11.12.tar.bz2
~/.wgetrcconfigfileforproxies:
http_proxy=<proxy>:<port>
ftp_proxy=<proxy>:<port>
proxy_user=<user>(ifany)
proxy_password=<passwd>(ifany)
56
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Downloadingkernelsourcepatches(1)
Assumingyoualreadyhavethelinuxx.y.<n1>version
Identifythepatchesyouneedonhttp://kernel.orgwithaweb
browser
Downloadthepatchfilesandtheirsignature:
Patchfrom2.6.10to2.6.11
wgetftp://ftp.kernel.org/pub/linux/kernel/v2.6/patch2.6.11.bz2
Patchfrom2.6.11to2.6.11.12(lateststablefixes)
wgethttp://www.kernel.org/pub/linux/kernel/v2.6/patch2.6.11.12.bz2
57
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Downloadingkernelsourcepatches(2)
Applythepatchesintherightorder:
cdlinux2.6.10/
bzcat../patch2.6.11.bz2|patchp1
bzcat../patch2.6.11.12.bz2|patchp1
cd..
mvlinux2.6.10linux2.6.11.12
58
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Anatomyofapatchfile
Apatchfileistheoutputofthediffcommand
diffcommandline
diffNrua/Makefileb/Makefile
a/Makefile2005030409:27:1508:00
Filedateinfo
+++b/Makefile2005030409:27:1508:00
@@1,7+1,7@@
Linenumbersinfiles
VERSION=2
Contextinfo:3linesbeforethechange
PATCHLEVEL=6
Usefultoapplyapatchwhenlinenumbers
SUBLEVEL=11
changed
EXTRAVERSION=
Removedline(s)ifany
+EXTRAVERSION=.1
Addedline(s)ifany
NAME=WoozyNumbat
#*DOCUMENTATION*
Contextinfo:3linesafterthechange
59
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Usingthepatchcommand
Thepatchcommandapplieschanges
tofilesinthecurrentdirectory:
Youcanreverse
apatch
withtheR
option
Makingchangestoexistingfiles
Creatingordeletingfilesanddirectories
patchusageexamples:
patchp<n><diff_file
catdiff_file|patchp<n>
bzcatdiff_file.bz2|patchp<n>
zcatdiff_file.gz|patchp<n>
Youcantestapatchwith
thedryrun
option
n:numberofdirectorylevelstoskipinthefilepaths
60
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
ApplyingaLinuxpatch
Linuxpatches...
Alwaystoapplytothex.y.<z1>version
Downloadableingzip
andbzip2(muchsmaller)compressedfiles.
Alwaysproducedforn=1
(that'swhateverybodydoes...doittoo!)
Youcanmakepatch30%
fasterbyusingsp1
insteadofp1
(silent)
Linuxpatchcommandlineexample:
cdlinux2.6.10
bzcat../patch2.6.11.bz2|patchp1
cd..;mvlinux2.6.10linux2.6.11
Testedonpatch2.6.23.bz2
Keeppatchfilescompressed:usefultochecktheirsignaturelater.
Youcanstillview(orevenedit)theuncompresseddatawithvi:
vipatch2.6.11.bz2(onthefly(un)compression)
61
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Accessingdevelopmentsources(1)
Kerneldevelopmentsourcesarenowmanagedwithgit:
http://kernel.org/pub/software/scm/git/
YoucanbrowseLinus'gittree(ifyoujustneedtocheckafewfiles):
http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux2.6.git;a=tree
Youcanalsodirectlyusegitonyourworkstation
Debian/Ubuntu:installthegitcorepackage
Ifyouarebehindaproxy,setUnixenvironmentvariablesdefining
proxysettings.Example:
exporthttp_proxy="proxy.server.com:8080"
exportftp_proxy="proxy.server.com:8080"
62
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Accessingdevelopmentsources(2)
Chooseagitdevelopmenttreeonhttp://git.kernel.org/
Getalocalcopy(clone)ofthistree.
Example(Linustree,theoneusedforLinuxstablereleases):
gitclonegit://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux2.6.git
Updateyourcopywheneverneeded(Linustreeexample):
cdlinux2.6
gitpull
Moredetailsavailable
onhttp://git.or.cz/orhttp://linux.yyz.us/githowto.html
63
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Onlinekerneldocumentation
http://freeelectrons.com/kerneldoc/
Providedforallrecentkernelreleases
Easierthandownloadingkernelsourcestoaccess
documentation
IndexedbyInternetsearchengines
Makeskernelpiecesofdocumentationeasiertofind!
Unlikemostothersitesofferingthisservicetoo,alsoincludesan
HTMLtranslationofkerneldocumentsintheDocBookformat.
Neverforgetdocumentationinthekernelsources!It'savery
valuablewayofgettinginformationaboutthekernel.
64
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
CompilingandbootingLinux
Kernelsourcemanagementtools
65
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Cscope
http://cscope.sourceforge.net/
Tooltobrowsesourcecode
(mainlyC,butalsoC++orJava)
SupportshugeprojectsliketheLinuxkernel
Takeslessthan1min.toindexLinux2.6.17
sources(fast!)
Canbeusedfromeditorslikevimandemacs.
InLinuxkernelsources,runitwith:
cscopeRk
(seemancscopefordetails)
Allowssearchingcodefor:
allreferencestoasymbol
globaldefinitions
functionscalledbyafunction
functionscallingafunction
textstring
regularexpressionpattern
afile
filesincludingafile
66
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Cscopescreenshot
67
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
KScope
http://kscope.sourceforge.net
AgraphicalfrontendtoCscope
MakesiteasytobrowseandedittheLinuxkernelsources
Candisplayafunctioncalltree
Niceeditingfeatures:symbolcompletion,spellingchecker,
automaticindentation...
Usageguidelines:
UsetheKernelsettingtoignorestandardCincludes.
Makesuretheprojectnamedoesn'tcontainblank
characters!
68
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
KScopescreenshots(1)
Symbols
incurrent
file
Project
files
Mainwindow
Querywindow
69
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
KScopescreenshots(2)
Calledfunctionstree
70
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
LXR:LinuxCrossReference
http://sourceforge.net/projects/lxr
Genericsourceindexingtool
andcodebrowser
Webserverbased
Veryeasyandfasttouse
Identifierortextsearchavailable
Veryeasytofindthedeclaration,
implementationorusagesofsymbols
SupportsCandC++
Supportshugecodeprojects
suchastheLinuxkernel
(264Minversion2.6.26).
Takessometimeandpatiencetosetup
(configuration,indexing,server
configuration).
Initialindexingveryslow:
Linux2.6.17:severalhoursonaserver
withanAMDSempron2200+CPU.
UsingKscopeistheeasiestandfastest
solutionformodifiedkernelsources.
Youdon'tneedtosetupLXRbyyourself.
Useourhttp://lxr.freeelectrons.comserver!
OtherserversavailableontheInternet:
http://freeelectrons.com/community/kernel/lxr/
ThismakesLXRthesimplestsolution
tobrowsestandardkernelsources.
71
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
LXRscreenshot
72
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
KetchupEasyaccesstokernelsources
http://www.selenic.com/ketchup/
Makesiteasytogetthelatestversionofagivenkernelsource
tree(2.4,2.6,2.6rc,2.6git,2.6mm,2.6rt...)
Onlydownloadstheneededpatches.
Revertspatcheswhenneededtoapplyamorerecentpatch.
Alsochecksthesignatureofsourcesandpatches.
73
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Ketchupexamples
Gettheversioninthecurrentdirectory:
>ketchupm
2.6.10
Upgradetothelateststableversion:
>ketchupG2.6tip
2.6.10>2.6.12.5
Applyingpatch2.6.11.bz2
Applyingpatch2.6.12.bz2
Applyingpatch2.6.12.5.bz2
Youcangetbackto2.6.8:
>ketchupG2.6.8
TheGoptionofketchup
disablessourcesignature
checking.
SeeourKernelsourcesannex
fordetailsaboutenabling
kernelsource
integritychecking.
74
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
PracticallabKernelsources
TimetostartLab1!
Getthesources
Applypatches
Getfamiliarwiththesources
Useakernelsourceindexingtool
75
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
CompilingandbootingLinux
Kernelconfiguration
76
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Kernelconfiguration
Defineswhatfeaturestoincludeinthekernel:
Storedinthe.configfileattherootofkernelsources.
Mostusefulcommandstocreatethisconfigfile:
make[xconfig|gconfig|menuconfig|oldconfig]
TomodifyakernelinaGNU/Linuxdistribution:
configfilesusuallyreleasedin/boot/,togetherwithkernel
images:/boot/config2.6.1711generic
Theconfigurationfilecanalsobefoundinthekernelitself:
>zcat/proc/config.gz
(ifenabledinGeneralSetup>Kernel.configsupport)
77
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
makexconfig
makexconfig
Themostcommongraphicalinterface
toconfigurethekernel.
Makesureyouread
help>introduction:usefuloptions!
Filebrowser:easiertoloadconfigurationfiles
Newsearchinterfacetolookforparameters
RequiredDebian/Ubuntupackages:
libqt3mtdev,g++
78
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
makexconfigscreenshot
79
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
makexconfigsearchinterface
Looksforakeyword
inthedescription
string
Allowstoselect
orunselectfound
parameters.
80
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Kernelconfigurationoptions
Compiledasamodule(separatefile)
CONFIG_ISO9660_FS=m
Driveroptions
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
Compiledstaticallyintothekernel
CONFIG_UDF_FS=y
81
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Corresponding.configfileexcerpt
#
#CDROM/DVDFilesystems
#
CONFIG_ISO9660_FS=m
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_UDF_FS=y
CONFIG_UDF_NLS=y
Sectionname
(helpstolocatesettingsintheinterface)
Allparametersareprefixed
withCONFIG_
#
#DOS/FAT/NTFilesystems
#
#CONFIG_MSDOS_FSisnotset
#CONFIG_VFAT_FSisnotset
CONFIG_NTFS_FS=m
#CONFIG_NTFS_DEBUGisnotset
CONFIG_NTFS_RW=y
82
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
makegconfig
makegconfig
NewGTKbased
graphicalconfiguration
interface.Functionality
similartothatofmake
xconfig.
Justlackingasearch
functionality.
RequiredDebian
packages:
libglade2dev
83
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
makemenuconfig
makemenuconfig
Usefulwhennographics
areavailable.Pretty
convenienttoo!
Sameinterfacefoundin
othertools:BusyBox,
buildroot...
RequiredDebian
packages:
libncursesdev
84
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
makeoldconfig
makeoldconfig
Neededveryoften!
Usefultoupgradea.configfilefromanearlierkernel
release
Issueswarningsforobsoletesymbols
Asksforvaluesfornewsymbols
Ifyouedita.configfilebyhand,it'sstrongly
recommendedtorunmakeoldconfigafterwards!
85
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
makeallnoconfig
makeallnoconfig
Onlysetsstronglyrecommendedsettingstoy.
Setsallothersettingston.
Veryusefulinembeddedsystemstoselectonlythe
minimumrequiredsetoffeaturesanddrivers.
Muchmoreconvenientthanunselectinghundredsof
featuresonebyone!
86
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Undoingconfigurationchanges
Afrequentproblem:
Afterchangingseveralkernelconfigurationsettings,
yourkernelnolongerworks.
Ifyoudon'trememberallthechangesyoumade,
youcangetbacktoyourpreviousconfiguration:
>cp.config.old.config
Alltheconfigurationinterfacesofthekernel
(xconfig,menuconfig,allnoconfig...)
keepthis.config.oldbackupcopy.
87
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
makehelp
makehelp
Listsallavailablemaketargets
Usefultogetareminder,ortolookforneworadvancedoptions!
88
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Customizingtheversionstring
Toidentifyyourkernelimagewithothersbuiltfromthe
samesources(butadifferentconfiguration),usethe
LOCALVERSIONsetting(inGeneralSetup)
Example:
#
#Generalsetup
#
CONFIG_LOCALVERSION="acme1"
Theunamercommand
(intherunningsystem)willreturn:
2.6.20acme1
89
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
CompilingandbootingLinux
Compilingthekernel
90
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Compilingandinstallingthekernel
Compilingstep
make
Installsteps
sudomakeinstall
sudomakemodules_install
91
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Compilingfasteronmultiprocessorhosts
Ifyouareusingaworkstationwithnprocessors,youmay
roughlydivideyourcompiletimebynbycompilingseveralfiles
inparallel
makej<n>
Runsseveraltargetsinparallel,wheneverpossible
Usingmakej2ormakej3onsingleprocessor
workstations.Thisdoesn'thelpmuch.Intheory,severalparallel
compilejobskeeptheprocessorbusywhileotherprocessesare
waitingforfilestobereadofwritten.Inpractice,youdon'tget
anysignificantspeedup(notmorethan10%),unlessyourI/Os
areveryslow.
92
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Compilingfasterwithccache
http://ccache.samba.org/
CompilercacheforCandC++,alreadyshippedbysome
distributions
Muchfasterwhencompilingthesamefileasecondtime!
Veryusefulwhen.configfilechangearefrequent.
Useitbyaddingaccacheprefix
totheCCandHOSTCCdefinitionsinMakefile:
CC=ccache$(CROSS_COMPILE)gcc
HOSTCC=ccachegcc
Performancebenchmarks:
63%:withaFedoraCore3configfile(manymodules!)
82%:withanembeddedLinuxconfigfile(muchfewermodules!)
93
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Kernelcompilingtips
Viewthefull(gcc,ld...)commandline:
makeV=1
Cleanupgeneratedfiles
(toforcerecompilingdrivers):
makeclean
Removeallgeneratedfiles
Caution:alsoremovesyour.configfile!
makemrproper
Alsoremoveeditorbackupandpatchrejectfiles:
(mainlytogeneratepatches):
makedistclean
94
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Generatedfiles
Createdwhenyourunthemakecommand
vmlinux
RawLinuxkernelimage,noncompressed.
arch/<arch>/boot/zImage
zlibcompressedkernelimage
(defaultimageonarm)
arch/<arch>/boot/bzImage
(defaultimageoni386)
Alsoazlibcompressedkernelimage.
Caution:bzmeansbigzippedbutnotbzip2compressed!
(bzip2compressionsupportonlyavailableoni386asatactical
patch.Notveryattractiveforsmallembeddedsystemsthough:
consumes1MBofRAMfordecompression).
95
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Filescreatedbymakeinstall
/boot/vmlinuz<version>
Compressedkernelimage.Sameastheonein
arch/<arch>/boot
/boot/System.map<version>
Storeskernelsymboladdresses
/boot/config<version>
Kernelconfigurationforthisversion
96
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Filescreatedbymakemodules_install(1)
/lib/modules/<version>/:Kernelmodules+extras
build/
Everythingneededtobuildmoremodulesforthiskernel:
Makefile,.configfile,modulesymbolinformation
(module.symVers),kernelheaders(include/andinclude/
asm/)
kernel/
Module.ko(KernelObject)files,inthesamedirectorystructure
asinthesources.
97
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Filescreatedbymakemodules_install(2)
/lib/modules/<version>/(continued)
modules.alias
Modulealiasesformoduleloadingutilities.Exampleline:
aliassoundservice?0snd_mixer_oss
modules.dep
Moduledependencies
(seetheLoadablekernelmodulessection)
modules.symbols
Tellswhichmoduleagivensymbolbelongsto.
Allthefilesinthisdirectoryaretextfiles.
Don'thesitatetohavealookbyyourself!
98
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Compilingthekernelinanutshell
makexconfig
make
sudomakeinstall
sudomakemodules_install
99
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
CompilingandbootingLinux
Linuxdevicefiles
100
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Characterdevicefiles
Accessedthroughasequentialflowofindividual
characters
Characterdevicescanbeidentifiedbytheirctype
(lsl):
crwrw1rootuucp4,64Feb232004/dev/ttyS0
crww1jdoetty136,1Feb232004/dev/pts/1
crw1rootroot13,32Feb232004/dev/input/mouse0
crwrwrw1rootroot1,3Feb232004/dev/null
Exampledevices:keyboards,mice,parallelport,IrDA,
Bluetoothport,consoles,terminals,sound,video...
101
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Blockdevicefiles
Accessedthroughdatablocksofagivensize.
Blockscanbeaccessedinanyorder.
Blockdevicescanbeidentifiedbytheirbtype(lsl):
brwrw1rootdisk3,1Feb232004hda1
brwrw1jdoefloppy2,0Feb232004fd0
brwrw1rootdisk7,0Feb232004loop0
brwrw1rootdisk1,1Feb232004ram1
brw1rootroot8,1Feb232004sda1
Exampledevices:hardorfloppydisks,ramdisks,loop
devices...
102
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Devicemajorandminornumbers
Asyoucouldseeinthepreviousexamples,
devicefileshave2numbersassociatedtothem:
Firstnumber:majornumber
Secondnumber:minornumber
Majorandminornumbersareusedbythekerneltobinda
drivertothedevicefile.Devicefilenamesdon'tmattertothe
kernel!
Tofindoutwhichdriveradevicefilecorrespondsto,
orwhenthedevicenameistoocryptic,
seeDocumentation/devices.txt.
103
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Devicefilecreation
Devicefilesarenotcreatedwhenadriverisloaded.
Theyhavetobecreatedinadvance:
sudomknod/dev/<device>[c|b]<major><minor>
Examples:
sudomknod/dev/ttyS0c464
sudomknod/dev/hda1b31
104
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
PracticallabConfiguringandcompiling
TimetostartLab2!
Configureyourkernel
Compileit
BootitonavirtualPC
Modifyarootfilesystemimageby
addingentriestothe/dev/
directory
105
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
CompilingandbootingLinux
Overallsystemstartup
106
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Linux2.4bootingsequence
Bootloader
ExecutedbythehardwareatafixedlocationinROM/Flash
Initializessupportforthedevicewherethekernelimageisfound(localstorage,
network,removablemedia)
LoadsthekernelimageinRAM
Executesthekernelimage(withaspecifiedcommandline)
Kernel
Uncompressesitself
Initializesthekernelcoreandstaticallycompileddrivers(neededtoaccesstheroot
filesystem)
Mountstherootfilesystem(specifiedbytherootkernelparameter)
Executesthefirstuserspaceprogram(specifiedbytheinitkernelparameter)
Firstuserspaceprogram
Configuresuserspaceandstartsupsystemservices
107
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Bootloader
ExecutedbythehardwareatafixedlocationinROM/Flash
Initializessupportforthedevicewheretheimagesarefound(localstorage,network,removablemedia)
LoadsthekernelimageinRAM
Executesthekernelimage(withaspecifiedcommandline)
Kernel
unchanged
Linux2.6bootingsequence
Uncompressesitself
Initializesthekernelcoreandstaticallycompileddrivers
Uncompressestheinitramfscpioarchiveincludedinthekernelfilecache(nomounting,nofilesystem).
Iffoundintheinitramfs,executesthefirstuserspaceprogram:/init
Userspace:/initscript(whatfollowsisjustatypicalscenario)
Runsuserspacecommandstoconfigurethedevice
(suchasnetworksetup,mounting/procand/sys...)
Mountsanewrootfilesystem.Switchtoit(switch_root)
Runs/sbin/init(orsometimesanew/linuxrcscript)
Userspace:/sbin/init
Runscommandstoconfigurethedevice(ifnotdoneyetintheinitramfs)
Startsupsystemservices(daemons,servers)anduserprograms
108
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Linux2.6bootingsequencewithinitrd
Bootloader
ExecutedbythehardwareatafixedlocationinROM/Flash
Initializessupportforthedevicewheretheimagesarefound(localstorage,network,removablemedia)
Loadsthekernelandinitramdisk(initrd)imagesinRAM
Executesthekernelimage(withaspecifiedcommandline)
Kernel
Uncompressesitself
Initializesstaticallycompileddrivers
Uncompressestheinitramfscpioarchiveincludedinthekernel.Mountsit.No/initexecutablefound.
Sofallsbacktotheoldwayoftryingtolocateandmountarootfilesystem.
Mountstherootfilesystemspecifiedbytherootkernelparameter(initrdinourcase)
Executesthefirstuserspaceprogram:usually/linuxrc
Userspace:/linuxrcscriptininitrd(whatfollowsisjustatypicalsequence)
Runsuserspacecommandstoconfigurethedevice(suchasnetworksetup,mounting/procand/sys...)
Loadskernelmodules(drivers)storedintheinitrd,neededtoaccessthenewrootfilesystem.
Mountsthenewrootfilesystem.Switchtoit(pivot_root)
Runs/sbin/init(orsometimesanew/linuxrcscript)
Userspace:/sbin/init
Runscommandstoconfigurethedevice(ifnotdoneyetintheinitrd)
Startsupsystemservices(daemons,servers)anduserprograms
109
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Linux2.4bootingsequencedrawbacks
Tryingtomountthefilesystemspecified
bytherootkernelparameteriscomplex:
Needdeviceandfilesystemdriverstobeloaded
Specifyingtherootfilesystemrequiresuglyblackmagic
devicenaming(suchas/dev/ram0,/dev/hda1...),
while/doesn'texistyet!
Canrequireacomplexinitializationtoimplementwithinthe
kernel.Examples:NFS(setupanIPaddress,connectto
theserver...),RAID(rootfilesystemonmultiplephysical
drives)...
Inanutshell:toomuchcomplexityinkernelcode!
110
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Extrainitramdiskdrawbacks
Initramdisksareimplementedasstandardblockdevices
Needaramdiskandfilesystemdriver
Fixedinsize:cannoteasilygrowinsize.
Anyfreespacecannotbereusedbyanythingelse.
Needstobecreatedandmodifiedlikeanyblockdevice:
formatting,mounting,editing,unmounting.
Rootpermissionsneeded.
Likeinanyblockdevice,filesarefirstreadfromthestorage,
andthencopiedtothefilecache.
SlowandduplicationinRAM!!!
111
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Initramfsfeaturesandadvantages(1)
Rootfilesystembuiltinthekernelimage
(embeddedasacompressedcpioarchive)
Veryeasytocreate(atkernelbuildtime).
Noneedforrootpermissions(formountandmknod).
Comparedtoinitramdisks,
just1filetohandleinthebootloader.
Justaplaincompressedcpioarchive
extractedinthefilecache.
Neitherneedsablocknorafilesystemdriver.
112
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Initramfsfeaturesandadvantages(2)
ramfs:implementedinthefilecache.NoduplicationinRAM,nofile
systemlayertomanage.Justusesthesizeofitsfiles.Cangrowifneeded.
Access
tofile
Regular
blockdevice
VirtualFile
System
File
cache
Filesystem
driver
Block(disk)
driver
VirtualFile
System
Filesystem
driver
Block
(ramdisk)
driver
Copy
Blockstorage
Access
tofile
RAM
Ramdisk
blockdevice
File
cache
Access
tofile
VirtualFile
System
ramfs
File
cache
Copy
Blockstorage
RAM
RAM
113
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Initramfsfeaturesandadvantages(3)
Loadedbythekernelearlier.
Moreinitializationcodemovedtouserspace!
Simplertomountcomplexfilesystemsfromflexible
userspacescriptsratherthanfromrigidkernelcode.More
complexitymovedouttouserspace!
Nomoremagicnamingoftherootdevice.
114
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Initramfsfeaturesandadvantages(4)
PossibletoaddnonGPLfiles(firmware,proprietarydrivers)
inthefilesystem.Thisisnotlinking,justfileaggregation
(notconsideredasaderivedworkbytheGPL).
Possibilitytoremovethesefileswhennolongerneeded.
Moretechnicaldetailsaboutinitramfs:
seeDocumentation/filesystems/ramfsrootfsinitramfs.txt
andDocumentation/earlyuserspace/READMEinkernelsources.
Seealsohttp://www.linuxdevices.com/articles/AT4017834659.htmlfora
niceoverviewofinitramfs(byRobLandley).
115
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Howtopopulateaninitramfs
UsingCONFIG_INITRAMFS_SOURCE
inkernelconfiguration(GeneralSetupsection)
Eithergiveanexistingcpioarchive
(filenameendingwith.cpio)
Orgiveadirectorytobearchived.
Anyotherregularfilewithbetakenasatextspecificationfile(see
nextpage).
YoucanbuildyourinitramfswithatinyClibrary
andthetinyexecutablesitships:
klibc:http://en.wikipedia.org/wiki/Klibc
116
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Initramfsspecificationfileexample
major minor
dir/dev75500
nod/dev/console64400c51
permissions
nod/dev/loop064400b70
dir/bin75510001000
file/bin/busybox/stuff/initramfs/busybox75500
slink/bin/shbusybox77700
dir/proc75500
dir/sys75500
dir/mnt75500
file/init/stuff/initramfs/init.sh75500
Noneedforrootuseraccess!
userid groupid
117
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Howtohandlecpioarchives
Usefulwhenyouwanttobuildthekernelwithareadymadecpio
archive,insteadoflettingthekerneldoitforyou.
Extracting:
cpioid<dir.cpio
Creating:
cddir
find.|cpioHnewco>../dir.cpio
NotethattheHnewcoptionisrequiredtogenerateacpioarchive
thatcanbeusedbytheLinuxkernel.
118
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Howtocreateaninitrd
Incaseyoureallyneedaninitrd(why?).
sudomkdir/mnt/initrd
ddif=/dev/zeroof=initrd.imgbs=1kcount=2048
mkfs.ext2Finitrd.img
sudomountoloopinitrd.img/mnt/initrd
Filltheramdiskcontents:BusyBox,modules,/linuxrcscript
MoredetailsintheFreeSoftwaretoolsforembeddedsystems
training!
sudoumount/mnt/initrd
gzipbestcinitrd.img>initrd
MoredetailsonDocumentation/initrd.txtinthekernel
sources!Alsoexplainspivotrooting.
119
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Bootingvariants
XIP(ExecuteInPlace)
Thekernelimageisdirectlyexecutedfromthestorage
CanbefasterandsaveRAM
However,thekernelimagecan'tbecompressed
Noinitramfs/initrd
Directlymountingthefinalrootfilesystem
(rootkernelcommandlineoption)
Nonewrootfilesystem
Runningthewholesystemfromtheinitramfs.
120
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
CompilingandbootingLinux
Bootloaders
121
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
x86bootloaders
LILO:LInuxLOad.OriginalLinuxbootloader.Nowrare.
http://freshmeat.net/projects/lilo/
Supports:x86
GRUB:GRandUnifiedBootloaderfromGNU.Morepowerful.
http://www.gnu.org/software/grub/
Supports:x86
SeeourGrubpresentationfordetails.
SYSLINUX:Utilitiesfornetworkandremovablemediabooting
http://syslinux.zytor.com
Supports:x86
122
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Genericbootloaders
DasUBoot:UniversalBootloaderfromDenxSoftware
Themostusedonarm.
http://www.denx.de/wiki/UBoot/WebHome
Supports:arm,ppc,mips,x86,m68k,nios...
SeeourUbootpresentationfordetails.
RedBoot:eCosbasedbootloaderfromRedHat
http://sources.redhat.com/redboot/
Supports:x86,arm,ppc,mips,sh,m68k...
uMon:MicroMonitorgeneralpurpose,multiOSbootloader
http://microcross.com/html/micromonitor.html
Supports:ARM,ColdFire,SH2,m68k,MIPS,PowerPC,
Xscale...
123
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
CompilingandbootingLinux
Kernelbooting
124
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Kernelcommandlineparameters
LikemostCprograms,theLinuxkernelaccepts
commandlinearguments
Kernelcommandlineargumentsarepartofthebootloader
configurationsettings.
Usefultomodifythebehaviorofthekernel
atboottime,withouthavingtorecompileit.
Usefultoperformadvancedkernelanddriverinitialization,
withouthavingtousecomplexuserspacescripts.
125
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Kernelcommandlineexample
HPiPAQh2200PDAbootingexample:
root=/dev/ram0\
rw\
init=/linuxrc\
console=ttyS0,115200n8\
console=tty0\
ramdisk_size=8192\
cachepolicy=writethrough
Rootfilesystem(firstramdisk)
Rootfilesystemmountingmode
Firstuserspaceprogram
Console(serial)
Otherconsole(framebuffer)
Miscparameters...
Hundredsofcommandlineparametersdescribedon
Documentation/kernelparameters.txt
126
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
UsefulnessofrootfsonNFS
Oncenetworkingworks,yourrootfilesystemcouldbeadirectory
onyourGNU/Linuxdevelopmenthost,exportedbyNFS(Network
FileSystem).Thisisveryconvenientforsystemdevelopment:
Makesitveryeasytoupdatefiles(drivermodulesinparticular)
ontherootfilesystem,withoutrebooting.Muchfasterthan
throughtheserialport.
Canhaveabigrootfilesystemevenifyoudon'thavesupport
forinternalorexternalstorageyet.
Therootfilesystemcanbehuge.Youcanevenbuildnative
compilertoolsandbuildallthetoolsyouneedonthetarget
itself(bettertocrosscompilethough).
127
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
NFSbootsetup(1)
Onthehost(NFSserver)
Addthebelowlinetoyour/etc/exportsfile:
/home/rootfs192.168.0.202(rw,no_root_squash,no_subtree_check)
clientaddress
NFSserveroptions
StartorrestartyourNFSserver(Example:Debian,Ubuntu)
/etc/init.d/nfskernelserverrestart
128
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
NFSbootsetup(2)
Onthetarget(NFSclient)
CompileyourkernelwithCONFIG_NFS_FS=y,
CONFIG_IP_PNP=y(configureIPatboottime)
andCONFIG_ROOT_NFS=y
Bootthekernelwiththebelowcommandlineoptions:
root=/dev/nfs
virtualdevice
ip=192.168.1.111:192.168.1.110:192.168.1.100:255.255.255.0:at91:eth0
localIPaddress serverIPaddress gateway
netmask
hostnamedevice
nfsroot=192.168.1.110:/home/nfsroot
NFSserverIPaddressDirectoryontheNFSserver
129
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Firstuserspaceprogram
Specifiedbytheinitkernelcommandlineparameter
Examples:init=/bin/shorinit=/sbin/init
Executedattheendofbootingbythekernel
Takescareofstartingallotheruserspaceprograms
(systemservicesanduserprograms).
Getsthe1processnumber(pid)
Parentorancestorofalluserspaceprograms
Thesystemwon'tletyoukillit.
130
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
/linuxrc
Programexecutedbydefaultwhenbootingfromaninit
ramdiskandnoinitparameterisgiventothekernel.
Ismostofthetimeashellscript,basedonaverylightweight
shellsuchasnashandbusyboxsh
Thisscriptcanimplementcomplextasks:detectingdrivers
toload,settingupnetworking,mountingpartitions,switching
toanewrootfilesystem...
131
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Theinitprogram
/sbin/initistheseconddefaultinitprogram
Takescareofstartingsystemservices,andeventuallythe
userinterfaces(sshd,Xserver...)
Alsotakescareofstoppingsystemservices
Lightweight,partialimplementationavailablethrough
BusyBox.
SeetheInitrunlevelsannexsectionformoredetailsabout
startingandstoppingsystemserviceswithinit.
However,simplestartupscriptsareoftensufficient
inembeddedsystems.
132
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
CompilingandbootingLinux
Crosscompilingthekernel
133
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Crosscompilingthekernel
WhenyoucompileaLinuxkernelforanotherCPUarchitecture
Muchfasterthancompilingnatively,whenthetargetsystemis
muchslowerthanyourGNU/Linuxworkstation.
MucheasierasdevelopmenttoolsforyourGNU/Linux
workstationaremucheasiertofind.
Tomakethedifferencewithanativecompiler,crosscompiler
executablesareprefixedbythenameofthetargetsystem,
architectureandsometimeslibrary.Examples:
mipslinuxgcc
m68klinuxuclibcgcc
armlinuxgnueabigcc
134
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Specifyingacrosscompiler(1)
TheCPUarchitectureandcrosscompilerprefixaredefinedthrough
theARCHandCROSS_COMPILEvariablesinthetoplevelMakefile.
TheMakefiledefinesCC=$(CROSS_COMPILE)gcc
SeecommentsinMakefilefordetails
TheeasiestsolutionistomodifytheMakefile.
Example,ARMplatform,crosscompiler:armlinuxgcc
ARCH
?=arm
CROSS_COMPILE ?=armlinux
135
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Specifyingacrosscompiler(2)
AnothersolutionistosetARCHandCROSS_COMPILE
throughthemakecommandline
Explanation:anyvariablesetthroughthemakecommandline
overridesanysettingintheMakefile.
Examples:
makeARCH=shCROSS_COMPILE=shlinuxxconfig
makeARCH=shCROSS_COMPILE=shlinux
makeARCH=shCROSS_COMPILE=shlinuxmodules_install
Bigdrawback:
Youshouldneverforgetthesesettingswhenyourunmake!
That'serrorproneandnotconvenientatall.
136
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Specifyingacrosscompiler(3)
Anothersolution:setARCHandCROSS_COMPILE
asenvironmentvariablesinyourterminal:
exportARCH=arm
exportCROSS_COMPILE=armlinux
Canbesetinprojectspecificenvironments.
NothardcodedintheMakefile.
Donotinterferewithpatches.
Youdon'tforgettosetthemwhenyourunanymake
command.
Caution:onlyapplytoshells
inwhichthesevariableshavebeenset.
137
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Configuringthekernel
makexconfig
Sameasinnativecompiling.
Don'tforgettosettherightboard/machinetype!
138
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Readymadeconfigfiles
assabet_defconfigintegrator_defconfigmainstone_defconfig
badge4_defconfigiq31244_defconfigmx1ads_defconfig
bast_defconfigiq80321_defconfigneponset_defconfig
cerfcube_defconfigiq80331_defconfignetwinder_defconfig
clps7500_defconfigiq80332_defconfigomap_h2_1610_defconfig
ebsa110_defconfigixdp2400_defconfigomnimeter_defconfig
edb7211_defconfigixdp2401_defconfigpleb_defconfig
enp2611_defconfigixdp2800_defconfigpxa255idp_defconfig
ep80219_defconfigixdp2801_defconfigrpc_defconfig
epxa10db_defconfigixp4xx_defconfigs3c2410_defconfig
footbridge_defconfigjornada720_defconfigshannon_defconfig
fortunet_defconfiglart_defconfigshark_defconfig
h3600_defconfiglpd7a400_defconfigsimpad_defconfig
h7201_defconfiglpd7a404_defconfigsmdk2410_defconfig
h7202_defconfiglubbock_defconfigversatile_defconfig
hackkit_defconfiglusl7200_defconfig
arch/arm/configsexample
139
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Usingreadymadeconfigfiles
Defaultconfigurationfilesavailableformanyboards/machines!
Checkifoneexistsinarch/<arch>/configs/foryourtarget.
Example:ifyoufoundanacme_defconfigfile,youcanrun:
makeacme_defconfig
Usingarch/<arch>/configs/isaverygoodgoodwayof
releasingadefaultconfigurationfileforagroupofusersor
developers.
Likeallmakecommands,youmust
runmake<machine>_defconfig
inthetoplevelsourcedirectory.
140
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Crosscompilingsetup
Example
IfyouhaveanARMcrosscompilingtoolchain
in/usr/local/arm/3.3.2/
YoujusthavetoaddittoyourUnixsearchpath:
exportPATH=/usr/local/arm/3.3.2/bin:$PATH
(Caution:thescopeofthisdefinitionislimitedtothecurrentshell).
Choosingatoolchain
SeetheDocumentation/Changesfileinthesources
fordetailsaboutminimumtoolversionsrequirements.
Moreabouttoolchains:FreeSoftwaretoolsforembeddedsystems
training:http://freeelectrons.com/training/devtools/
141
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Buildingthekernel
Run
make
Copy
arch/<arch>/boot/zImage
tothetargetstorage
Youcancustomizearch/<arch>/boot/install.shso
thatmakeinstalldoesthisautomaticallyforyou.
makeINSTALL_MOD_PATH=<dir>/modules_install
andcopy<dir>/to/lib/modules/onthetargetstorage
142
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Crosscompilingsummary
EditMakefile:setARCHandCROSS_COMPILE
Getthedefaultconfigurationforyourmachine:
make<machine>_defconfig(ifexistingin
arch/<arch>/configs)
Refinetheconfigurationsettingsaccordingtoyourrequirements:
makexconfig
AddthecrosscompilerpathtoyourPATHenvironmentvariable
Compilethekernel:make
Copythekernelimagefromarch/<arch>/boot/tothetarget
Copymodulestoadirectorywhichyoureplicateonthetarget:
makeINSTALL_MOD_PATH=<dir>modules_install
143
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
PracticallabCrosscompiling
TimetostartLab3!
Setupacrosscompiling
environment
ConfigurethekernelMakefile
accordingly
Crosscompilethekernelforanarm
targetplatform
144
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Driverdevelopment
Loadablekernelmodules
145
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Loadablekernelmodules
Modules:addagivenfunctionalitytothekernel
(drivers,filesystemsupport,andmanyothers).
Canbeloadedandunloadedatanytime,onlywhen
theirfunctionalityisneed.Onceloaded,havefull
accesstothewholekerneladdressspace.No
particularprotection.
Usefultokeepthekernelimagesizetotheminimum
(essentialinGNU/LinuxdistributionsforPCs).
Usefultodeliverbinaryonlydrivers(badidea)
withouthavingtorebuildthekernel.
Modulesmakeiteasytodevelopdriverswithout
rebooting:load,test,unload,rebuild,load...
146
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Symbolsexportedtomodules
Fromakernelmodule,
onlyalimitednumberofkernelfunctionscanbecalled
Functionsandvariableshavetobeexplicitlyexported
bythekerneltobevisiblefromakernelmodule
Twomacrosareusedinthekernel
toexportfunctionsandvariables:
EXPORT_SYMBOL(symbolname),whichexportsa
functionorvariabletoallmodules
EXPORT_SYMBOL_GPL(symbolname),whichexportsa
functionorvariableonlytoGPLmodules
Anormaldrivershouldnotneedanynonexportedfunction.
147
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Moduledependencies
Moduledependenciesstoredin
/lib/modules/<version>/modules.dep
Theyareautomaticallycomputedduringkernelbuildingfrom
moduleexportedsymbols.module2dependsonmodule1if
module2
usesasymbolexportedbymodule1.
Example:usb_storagedependsonusbcore,
becauseitusessomeofthefunctionsexportedbyusbcore.
Youcanalsoupdatethemodules.depfile
byyourself,byrunning(asroot):
depmoda[<version>]
148
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
hellomodule
/*hello.c*/
#include<linux/init.h>
#include<linux/module.h>
#include<linux/kernel.h>
staticint__inithello_init(void)
{
printk(KERN_ALERT"Goodmorrow");
printk(KERN_ALERT"tothisfairassembly.\n");
return0;
}
staticvoid__exithello_exit(void)
{
printk(KERN_ALERT"Alas,poorworld,whattreasure");
printk(KERN_ALERT"hastthoulost!\n");
}
__init:
removedafterinitialization
(statickernelormodule).
__exit:discardedwhen
modulecompiledstatically
intothekernel.
module_init(hello_init);
module_exit(hello_exit);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("Greetingmodule");
MODULE_AUTHOR("WilliamShakespeare");
Exampleavailableonhttp://freeelectrons.com/doc/c/hello.c
149
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Modulelicenseusefulness
Usedbykerneldeveloperstoidentifyissuescomingfrom
proprietarydrivers,whichtheycan'tdoanythingabout
(Taintedkernelnoticeinkernelcrashesandoopses).
Usefulforuserstocheckthattheirsystemis100%free
(check/proc/sys/kernel/tainted)
UsefulforGNU/Linuxdistributors
fortheirreleasepolicychecks.
150
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Possiblemodulelicensestrings
Availablelicensestringsexplainedininclude/linux/module.h
GPL
GNUPublicLicensev2or
later
DualBSD/GPL
GNUPublicLicensev2orBSD
GPLv2
GNUPublicLicensev2
DualMPL/GPL
GNUPublicLicensev2
orMozilla
GPLandadditional
rights
Proprietary
Nonfreeproducts
DualMIT/GPL
GNUPublicLicensev2or
MIT
151
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Compilingamodule
ThebelowMakefileshouldbereusableforanyLinux2.6
module.
Justrunmaketobuildthehello.kofile
Caution:makesurethereisa[Tab]characteratthe
beginningofthe$(MAKE)line(makesyntax)
#Makefileforthehellomodule
[Tab]!
(nospaces)
objm:=hello.o
KDIR:=/lib/modules/$(shellunamer)/build
PWD:=$(shellpwd)
default:
$(MAKE)C$(KDIR)SUBDIRS=$(PWD)modules
Either
fullkernel
sourcedirectory
(configuredand
compiled)
orjustkernel
headersdirectory
(minimum
needed)
Exampleavailableonhttp://freeelectrons.com/doc/c/Makefile
152
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Kernellog
Ofcourse,thekerneldoesn'tstoreitslogintoafile!
Filesbelongtouserspace.
Thekernelkeepsprintkmessagesinacircularbuffer
(sothatdoesn'tconsumemorememorywithmanymessages)
Kernellogmessagescanbeaccessedfromuserspacethrough
systemcalls,orthrough/proc/kmsg
Kernellogmessagesarealsodisplayedinthesystemconsole.
153
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Accessingthekernellog
Manywaysareavailable!
Watchthesystemconsole
syslogd/klogd
Daemongatheringkernel
messages
in/var/log/messages
Followchangesbyrunning:
tailf/var/log/messages
Caution:thisfilegrows!
Uselogrotatetocontrolthis
cat/proc/kmsg
Waitsforkernelmessagesand
displaysthem.
Usefulwhennoneoftheabove
userspaceprogramsare
available(tinysystem)
dmesg(diagnosticmessage)
Foundinallsystems
Displaysthekernellogbuffer
154
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Usingthemodule
Loadthemodule:
sudoinsmod./hello.ko
Youwillseethefollowinginthekernellog:
Goodmorrow
tothisfairassembly
Nowremovethemodule:
sudormmodhello
Youwillsee:
Alas,poorworld,whattreasure
hastthoulost!
155
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Understandingmoduleloadingissues
Whenloadingamodulefails,
insmodoftendoesn'tgiveyouenoughdetails!
Detailsareavailableinthekernellog.
Example:
>sudoinsmod./intr_monitor.ko
insmod:errorinserting'./intr_monitor.ko':1
Deviceorresourcebusy
>dmesg
[17549774.552000]Failedtoregisterhandlerfor
irqchannel2
156
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Moduleutilities(1)
modinfo<module_name>
modinfo<module_path>.ko
Getsinformationaboutamodule:parameters,license,
descriptionanddependencies.
Veryusefulbeforedecidingtoloadamoduleornot.
sudoinsmod<module_path>.ko
Triestoloadthegivenmodule.
157
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Moduleutilities(2)
sudomodprobe<module_name>
Mostcommonusageofmodprobe:triestoloadallthe
modulesthegivenmoduledependson,andthenthis
module.Lotsofotheroptionsareavailable.
lsmod
Displaysthelistofloadedmodules
Compareitsoutputwiththecontentsof
/proc/modules!
158
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Moduleutilities(3)
sudormmod<module_name>
Triestoremovethegivenmodule
sudomodprober<module_name>
Triestoremovethegivenmoduleandalldependent
modules(whicharenolongerneededafter
themoduleremoval)
159
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Createyourmoduleswithkdevelop
http://kdevelop.orgAvailableinmostdistros.
Makesiteasytocreate
amodulecodeskeleton
fromareadymade
template.
Canalsobeusedto
compileyourmodule.
160
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Driverdevelopment
Moduleparameters
161
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
hellomodulewithparameters
/*hello_param.c*/
#include<linux/init.h>
#include<linux/module.h>
#include<linux/moduleparam.h>
MODULE_LICENSE("GPL");
Thanksto
JonathanCorbet
fortheexample!
/*Acoupleofparametersthatcanbepassedin:howmanytimeswesay
hello,andtowhom*/
staticchar*whom="world";
module_param(whom,charp,0);
staticinthowmany=1;
module_param(howmany,int,0);
staticint__inithello_init(void)
{
inti;
for(i=0;i<howmany;i++)
printk(KERN_ALERT"(%d)Hello,%s\n",i,whom);
return0;
}
staticvoid__exithello_exit(void)
{
printk(KERN_ALERT"Goodbye,cruel%s\n",whom);
}
module_init(hello_init);
module_exit(hello_exit);
Exampleavailableonhttp://freeelectrons.com/doc/c/hello_param.c
162
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Passingmoduleparameters
Throughinsmod:
sudoinsmod./hello_param.kohowmany=2whom=universe
Throughmodprobe:
Setparametersin/etc/modprobe.conforinanyfilein
/etc/modprobe.d/:
optionshello_paramhowmany=2whom=universe
Throughthekernelcommandline,
whenthemoduleisbuiltstaticallyintothekernel:
optionshello_param.howmany=2hello_param.whom=universe
modulename
moduleparametername
moduleparametervalue
163
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Declaringamoduleparameter
#include<linux/moduleparam.h>
module_param(
name,
/*nameofanalreadydefinedvariable*/
type,
/*eitherbyte,short,ushort,int,uint,long,
ulong,charp,orbool.
(checkedatcompiletime!)*/
perm
/*for/sys/module/<module_name>/parameters/<param>
0:nosuchmoduleparametervaluefile*/
);
Example
intirq=5;
module_param(irq,int,S_IRUGO);
164
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Declaringamoduleparameterarray
#include<linux/moduleparam.h>
module_param_array(
name,
/*nameofanalreadydefinedarray*/
type,
/*sameasinmodule_param*/
num,
/*numberofelementsinthearray,orNULL(nocheck?)*/
perm
/*sameasinmodule_param*/
);
Example
staticintbase[MAX_DEVICES]={0x820,0x840};
module_param_array(base,int,NULL,0);
165
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Driverdevelopment
Addingsourcestothekerneltree
166
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Newdriverinkernelsources(1)
Toaddanewdrivertothekernelsources:
Addyournewsourcefiletotheappropriatesourcedirectory.
Example:drivers/usb/serial/navman.c
Describetheconfigurationinterfaceforyournewdriver
byaddingthefollowinglinestotheKconfigfileinthisdirectory:
configUSB_SERIAL_NAVMAN
tristate"USBNavmanGPSdevice"
dependsonUSB_SERIAL
help
Tocompilethisdriverasamodule,chooseMhere:the
modulewillbecallednavman.
167
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Newdriverinkernelsources(2)
AddalineintheMakefilefilebasedontheKconfigsetting:
obj$(CONFIG_USB_SERIAL_NAVMAN)+=navman.o
Runmakexconfigandseeyournewoptions!
Runmakeandyournewfilesarecompiled!
SeeDocumentation/kbuild/fordetails
168
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
HowtocreateLinuxpatches
Downloadthelatestkernelsources
Makeacopyofthesesources:
rsyncalinux2.6.9rc2/linux2.6.9rc2patch/
Applyyourchangestothecopiedsources,andtestthem.
Runmakedistcleantokeeponlysourcefiles.
Createapatchfile:
diffNurplinux2.6.9rc2/\
linux2.6.9rc2patch/>patchfile
Alwayscomparethewholesourcestructures
(suitableforpatchp1)
Patchfilename:shouldrecallwhatthepatchisabout.
ThankstoNicolasRougier(Copyright2003,
http://webloria.loria.fr/~rougier/)fortheTuximage
169
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
PracticallabWritingmodules
TimetostartLab4!
Writeakernelmodulewith
parameters
Setuptheenvironmenttocompileit
Accesskernelinternals
Adda/procinterface
Addthemodulesourcestothe
kernelsourcetree
Createakernelsourcepatch
170
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Driverdevelopment
Memorymanagement
171
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Physicalandvirtualmemory
Physicaladdressspace
Virtualaddressspaces
0xFFFFFFFF
0xFFFFFFFFF
I/Omemory3
I/Omemory2
I/Omemory1
Flash
0xFFFFFFFF
0xC0000000
Memory
Management
Unit
MMU
Process1
0x00000000
0x00000000
CPU
0xFFFFFFFF
RAM1
RAM0
Kernel
Kernel
0xC0000000
Alltheprocesseshavetheir
ownvirtualaddressspace,and
runasiftheyhadaccesstothe
wholeaddressspace.
Process2
0x00000000
0x00000000
172
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
kmallocandkfree
Basicallocators,kernelequivalentsofglibc'smallocand
free.
#include<linux/slab.h>
staticinlinevoid*kmalloc(size_tsize,intflags);
size:numberofbytestoallocate
flags:priority(explainedinafewpages)
voidkfree(constvoid*objp);
Example:(drivers/infiniband/core/cache.c)
structib_update_work*work;
work=kmalloc(sizeof*work,GFP_ATOMIC);
...
kfree(work);
173
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
kmallocfeatures
Quick(unlessit'sblockedwaitingformemorytobefreed).
Doesn'tinitializetheallocatedarea.
TheallocatedareaiscontiguousinphysicalRAM.
Allocatesby2nsizes,andusesafewmanagementbytes.
So,don'taskfor1024whenyouneed1000!You'dget2048!
Caution:driversshouldn'ttrytokmalloc
morethan128KB(upperlimitinsomearchitectures).
Minimumallocation:32or64bytes(pagesizedependent).
174
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Mainkmallocflags(1)
Definedininclude/linux/gfp.h(GFP:__get_free_pages)
GFP_KERNEL
Standardkernelmemoryallocation.Mayblock.Fineformost
needs.
GFP_ATOMIC
RAMallocatedfromcodewhichisnotallowedtoblock(interrupt
handlers)orwhichdoesn'twanttoblock(criticalsections).Never
blocks.
GFP_USER
Allocatesmemoryforuserprocesses.Mayblock.Lowestpriority.
175
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Mainkmallocflags(2)
Extraflags(canbeaddedwith|)
__GFP_DMAorGFP_DMA
AllocateinDMAzone
__GFP_ZERO
Returnsazeroedpage.
__GFP_NOFAIL
Mustnotfail.Nevergives
up.Caution:useonlywhen
mandatory!
__GFP_NORETRY
Ifallocationfails,doesn'ttry
togetfreepages.
Example:
GFP_KERNEL|__GFP_DMA
Note:almostonly
__GFP_DMAorGFP_DMA
usedindevicedrivers.
176
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Relatedallocationfunctions
Again,namessimilartothoseofClibraryfunctions
staticinlinevoid*kzalloc(
size_tsize,gfp_tflags);
Zeroestheallocatedbuffer.
staticinlinevoid*kcalloc(
size_tn,size_tsize,gfp_tflags);
Allocatesmemoryforanarrayofnelementsofsizesize,
andzeroesitscontents.
void*__must_checkkrealloc(
constvoid*,size_t,gfp_t);
Changesthesizeofthegivenbuffer.
177
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Availableallocators
Memoryisallocatedusingslabs(groupsofoneormorecontinuouspages
fromwhichobjectsareallocated).Severalcompatibleslaballocatorsare
available:
SLAB:original,wellprovenallocatorinLinux2.6.
SLOB:muchsimpler.Morespaceefficientbutdoesn'tscalewell.Saves
afewhundredsofKBinsmallsystems(dependson
CONFIG_EMBEDDED)
SLUB:thenewdefaultallocatorsince2.6.23,simplerthanSLAB,
scalingmuchbetter(inparticularforhugesystems)andcreatingless
fragmentation.
178
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Slabcachesandmemorypools
Slabcaches:makeitpossibletoallocatemultiple
objectsofthesamesize,withoutwastingRAM.
Sofar,mainlyusedincoresubsystems,
butnotmuchindevicedrivers
(exceptUSBandSCSIdrivers)
Memorypools:poolsofpreallocatedobjects,
toincreasethechancesofallocationstosucceed.
Oftenusedwithfilecaches.
SeeourSlabcachesandmemorypoolsannexfordetails.
179
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Allocatingbypages
MoreappropriatewhenyouneedbigslicesofRAM:
Apageisusually4K,butcanbemadegreaterinsomearchitectures
(sh,mips:4,8,16or64K,butnotconfigurableini386orarm).
unsignedlongget_zeroed_page(intflags);
Returnsapointertoafreepageandfillsitupwithzeros
unsignedlong__get_free_page(intflags);
Same,butdoesn'tinitializethecontents
unsignedlong__get_free_pages(intflags,
unsignedintorder);
ReturnsapointeronanareaofseveralcontiguouspagesinphysicalRAM.
order:log2(<number_of_pages>)
Ifvariable,canbecomputedfromthesizewiththeget_orderfunction.
Maximum:8192KB(MAX_ORDER=11ininclude/linux/mmzone.h),
exceptinafewarchitectureswhenoverwrittenwithCONFIG_FORCE_MAX_ZONEORDER.
180
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Freeingpages
voidfree_page(unsignedlongaddr);
voidfree_pages(unsignedlongaddr,
unsignedintorder);
Needtousethesameorderasinallocation.
181
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
vmalloc
vmalloccanbeusedtoobtaincontiguousmemoryzones
invirtualaddressspace(evenifpagesmaynotbe
contiguousinphysicalmemory).
void*vmalloc(unsignedlongsize);
voidvfree(void*addr);
182
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Memoryutilities
void*memset(void*s,intc,size_tcount);
Fillsaregionofmemorywiththegivenvalue.
void*memcpy(void*dest,
constvoid*src,
size_tcount);
Copiesoneareaofmemorytoanother.
Usememmovewithoverlappingareas.
LotsoffunctionsequivalenttostandardClibraryonesdefined
ininclude/linux/string.h
andininclude/linux/kernel.h(sprintf,etc.)
183
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
MemorymanagementSummary
Smallallocations
kmalloc,kzalloc
(andkfree!)
Slabcachesandmemory
pools
Biggerallocations
__get_free_page[s],
get_zeroed_page,
free_page[s]
vmalloc,vfree
Libclikememoryutilities
memset,memcopy,memmove
...
184
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Driverdevelopment
I/Omemoryandports
185
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
RequestingI/Oports
/proc/ioportsexample(x86)
0000001f:dma1
00200021:pic1
00400043:timer0
00500053:timer1
0060006f:keyboard
00700077:rtc
0080008f:dmapagereg
00a000a1:pic2
00c000df:dma2
00f000ff:fpu
0100013f:pcmcia_socket0
01700177:ide1
01f001f7:ide0
03760376:ide1
0378037a:parport0
03c003df:vga+
03f603f6:ide0
03f803ff:serial
0800087f:0000:00:1f.0
08000803:PM1a_EVT_BLK
08040805:PM1a_CNT_BLK
0808080b:PM_TMR
08200820:PM2_CNT_BLK
0828082f:GPE0_BLK
...
structresource*request_region(
unsignedlongstart,
unsignedlonglen,
char*name);
Triestoreservethegivenregionandreturns
NULLifunsuccessful.Example:
request_region(0x0170,8,"ide1");
voidrelease_region(
unsignedlongstart,
unsignedlonglen);
Seeinclude/linux/ioport.hand
kernel/resource.c
186
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Reading/writingonI/Oports
Theimplementationofthebelowfunctions
andtheexactunsignedtypecanvaryfromplatformtoplatform!
bytes
unsignedinb(unsignedport);
voidoutb(unsignedcharbyte,unsignedport);
words
unsignedinw(unsignedport);
voidoutw(unsignedcharbyte,unsignedport);
"long"integers
unsignedinl(unsignedport);
voidoutl(unsignedcharbyte,unsignedport);
187
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Reading/writingstringsonI/Oports
OftenmoreefficientthanthecorrespondingCloop,
iftheprocessorsupportssuchoperations!
bytestrings
voidinsb(unsignedport,void*addr,unsignedlongcount);
voidoutsb(unsignedport,void*addr,unsignedlongcount);
wordstrings
voidinsw(unsignedport,void*addr,unsignedlongcount);
voidoutsw(unsignedport,void*addr,unsignedlongcount);
longstrings
voidinsl(unsignedport,void*addr,unsignedlongcount);
voidoutsl(unsignedport,void*addr,unsignedlongcount);
188
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
RequestingI/Omemory
/proc/iomemexample
Equivalentfunctionswiththesame
000000000009efff:SystemRAM
0009f0000009ffff:reserved
000a0000000bffff:VideoRAMarea
000c0000000cffff:VideoROM
000f0000000fffff:SystemROM
001000003ffadfff:SystemRAM
001000000030afff:Kernelcode
0030b000003b4bff:Kerneldata
3ffae0003fffffff:reserved
40000000400003ff:0000:00:1f.1
4000100040001fff:0000:02:01.0
4000100040001fff:yenta_socket
4000200040002fff:0000:02:01.1
4000200040002fff:yenta_socket
40400000407fffff:PCICardBus#03
4080000040bfffff:PCICardBus#03
40c0000040ffffff:PCICardBus#07
41000000413fffff:PCICardBus#07
a0000000a0000fff:pcmcia_socket0
a0001000a0001fff:pcmcia_socket1
e0000000e7ffffff:0000:00:00.0
e8000000efffffff:PCIBus#01
e8000000efffffff:0000:01:00.0
...
interface
structresource*request_mem_region(
unsignedlongstart,
unsignedlonglen,
char*name);
voidrelease_mem_region(
unsignedlongstart,
unsignedlonglen);
189
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
MappingI/Omemoryinvirtualmemory
ToaccessI/Omemory,driversneedtohaveavirtual
addressthattheprocessorcanhandle.
Theioremapfunctionssatisfythisneed:
#include<asm/io.h>;
void*ioremap(unsignedlongphys_addr,
unsignedlongsize);
voidiounmap(void*address);
Caution:checkthatioremapdoesn'treturnaNULL
address!
Notethatanioremap_nocachefunctionexists.
ThisdisablestheCPUcacheatthegivenaddressrange.
190
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Differenceswithstandardmemory
Readsandwritesonmemorycanbecached
Thecompilermaychoosetowritethevalueinacpu
register,andmayneverwriteitinmainmemory.
Thecompilermaydecidetooptimizeorreorderreadand
writeinstructions.
191
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
AvoidingI/Oaccessissues
CachingonI/Oportsormemoryalreadydisabled,eitherbythe
hardwareorbyLinuxinitcode.
UsethevolatilestatementinyourCcodetopreventthe
compilerfromusingregistersinsteadofwritingtomemory.
Memorybarriersaresuppliedtoavoidreordering
Hardwareindependent
#include<asm/kernel.h>
voidbarrier(void);
Hardwaredependent
#include<asm/system.h>
voidrmb(void);
Onlyimpactsthebehaviorofthe
voidwmb(void);
compiler.Doesn'tpreventreordering voidmb(void);
intheprocessor!
Safeonallarchitectures!
192
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
AccessingI/Omemory
Directlyreadingfromorwritingtoaddressesreturnedbyioremap
(pointerdereferencing)maynotworkonsomearchitectures.
Usethebelowfunctionsinstead.Theyarealwaysportableand
safe:
unsignedintioread8(void*addr);(samefor16and32)
voidiowrite8(u8value,void*addr);(samefor16and32)
Toreadorwriteaseriesofvalues:
voidioread8_rep(void*addr,void*buf,unsignedlongcount);
voidiowrite8_rep(void*addr,constvoid*buf,unsignedlongcount);
Otherusefulfunctions:
voidmemset_io(void*addr,u8value,unsignedintcount);
voidmemcpy_fromio(void*dest,void*source,unsignedintcount);
voidmemcpy_toio(void*dest,void*source,unsignedintcount);
Note:manydriversstilluseoldfunctionsinstead:
readb,readl,readw,writeb,writel,writew
193
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
/dev/mem
Usedtoprovideuserspaceapplications
withdirectaccesstophysicaladdresses.
Usage:open/dev/memandreadorwriteatgivenoffset.
Whatyoureadorwriteisthevalue
atthecorrespondingphysicaladdress.
UsedbyapplicationssuchastheXserver
towritedirectlytodevicememory.
Since2.6.26(x86onlysofar):onlynonRAMcanbe
accessed,unlessexplicitlyconfiguredotherwise(forsecurity
reasons).
194
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Driverdevelopment
Characterdrivers
195
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Usefulnessofcharacterdrivers
Exceptforstoragedevicedrivers,mostdriversfordeviceswith
inputandoutputflowsareimplementedascharacterdrivers.
So,mostdriversyouwillfacewillbecharacterdrivers
Youwillregretifyousleepduringthispart!
196
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Creatingacharacterdriver
Userspace
Userspaceneeds
Thenameofadevicefilein/devto
interactwiththedevicedriverthrough
regularfileoperations(open,read,write,
close...)
Read
buffer
read
write
Thekernelneeds
Copytouser
major/minor
Read
handler
Write
handler
Copyfromuser
/dev/foo
Toknowwhichdriverisinchargeofdevice
fileswithagivenmajor/minornumberpair
Foragivendriver,tohavehandlers(file
operations)toexecutewhenuserspace
opens,reads,writesorclosesthedevice
file.
Write
string
Devicedriver
Kernelspace
197
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Declaringacharacterdriver
Devicenumberregistration
Needtoregisteroneormoredevicenumbers(major/minor
pairs),dependingonthenumberofdevicesmanagedbythe
driver.
Needtofindfreeones!
Fileoperationsregistration
Needtoregisterhandlerfunctionscalledwhenuserspace
programsaccessthedevicefiles:open,read,write,
ioctl,close...
198
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Informationonregistereddevices
Registereddevicesarevisiblein/proc/devices:
Characterdevices:
1mem
4/dev/vc/0
4tty
4ttyS
5/dev/tty
5/dev/console
5/dev/ptmx
6lp
10misc
13input
14sound
...
Blockdevices:
1ramdisk
3ide0
8sd
9md
22ide1
65sd
66sd
67sd
68sd
Major
number
Canbeusedto
findfreemajor
numbers
Registered
name
199
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
dev_tdatatype
Kerneldatatypetorepresentamajor/minornumberpair
Alsocalledadevicenumber.
Definedin<linux/kdev_t.h>
Linux2.6:32bitsize(major:12bits,minor:20bits)
Macrotocreatethedevicenumber:
MKDEV(intmajor,intminor);
Macrotoextracttheminorandmajornumbers:
MAJOR(dev_tdev);
MINOR(dev_tdev);
200
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Allocatingfixeddevicenumbers
#include<linux/fs.h>
intregister_chrdev_region(
dev_tfrom,
/*Startingdevicenumber*/
unsignedcount,
/*Numberofdevicenumbers*/
constchar*name); /*Registeredname*/
Returns0iftheallocationwassuccessful.
Example
if(register_chrdev_region(MKDEV(202,128),
acme_count,acme)){
printk(KERN_ERRFailedtoallocatedevicenumber\n);
...
201
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Dynamicallocationofdevicenumbers
Safer:havethekernelallocatefreenumbersforyou!
#include<linux/fs.h>
intalloc_chrdev_region(
dev_t*dev,
/*Output:startingdevicenumber*/
unsignedbaseminor, /*Startingminornumber,usually0*/
unsignedcount,
/*Numberofdevicenumbers*/
constchar*name); /*Registeredname*/
Returns0iftheallocationwassuccessful.
Example
if(alloc_chrdev_region(&acme_dev,0,acme_count,acme)){
printk(KERN_ERRFailedtoallocatedevicenumber\n);
...
202
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Creatingdevicefiles
Issue:youcannolongercreate/deventriesinadvance!
Youhavetocreatethemontheflyafterloadingthedriver
accordingtotheallocatedmajornumber.
Trick:thescriptloadingthemodulecanthenuse
/proc/devices:
module=foo;name=foo;device=foo
rmf/dev/$device
insmod$module.ko
major=`awk"\\$2==\"$name\"{print\\$1}"/proc/devices`
mknod/dev/$devicec$major0
Better:useudevtocreatethedevicefileautomatically.
Seeourpresentationonudev.
203
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Fileoperations(1)
Beforeregisteringcharacterdevices,youhavetodefine
file_operations(calledfops)forthedevicefiles.
Herearethemainones:
int(*open)(
structinode*,/*Correspondstothedevicefile*/
structfile*);/*Correspondstotheopenfiledescriptor*/
Calledwhenuserspaceopensthedevicefile.
int(*release)(
structinode*,
structfile*);
Calledwhenuserspaceclosesthefile.
204
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Thefilestructure
Iscreatedbythekernelduringtheopencall.
Representsopenfiles.
mode_tf_mode;
Thefileopeningmode(FMODE_READand/orFMODE_WRITE)
loff_tf_pos;
Currentoffsetinthefile.
structfile_operations*f_op;
Allowstochangefileoperationsfordifferentopenfiles!
structdentry*f_dentry
Usefultogetaccesstotheinode:f_dentry>d_inode.
205
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Fileoperations(2)
ssize_t(*read)(
structfile*,
/*Openfiledescriptor*/
__userchar*,
/*Userspacebuffertofillup*/
size_t,
/*Sizeoftheuserspacebuffer*/
loff_t*);
/*Offsetintheopenfile*/
Calledwhenuserspacereadsfromthedevicefile.
ssize_t(*write)(
structfile*,
__userconstchar*,
/*Openfiledescriptor*/
/*Userspacebuffertowrite
tothedevice*/
size_t,
/*Sizeoftheuserspacebuffer*/
loff_t*);
/*Offsetintheopenfile*/
Calledwhenuserspacewritestothedevicefile.
206
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Exchangingdatawithuserspace(1)
Indrivercode,youcan'tjustmemcpybetween
anaddresssuppliedbyuserspaceand
theaddressofabufferinkernelspace!
Correspondtocompletelydifferent
addressspaces(thankstovirtualmemory)
Theuserspaceaddressmaybeswappedouttodisk
Theuserspaceaddressmaybeinvalid
(userspaceprocesstryingtoaccessunauthorizeddata)
207
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Exchangingdatawithuserspace(2)
Youmustusededicatedfunctionssuchasthefollowingones
inyourreadandwritefileoperationscode:
include<asm/uaccess.h>
unsignedlongcopy_to_user(void__user*to,
constvoid*from,
unsignedlongn);
unsignedlongcopy_from_user(void*to,
constvoid__user*from,
unsignedlongn);
Makesurethatthesefunctionsreturn0!
Anotherreturnvaluewouldmeanthattheyfailed.
208
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Fileoperations(3)
int(*ioctl)(structinode*,structfile*,
unsignedint,unsignedlong);
Canbeusedtosendspecificcommandstothedevice,whichare
neitherreadingnorwriting(e.g.changingthespeedofaserial
port,settingvideooutputformat,queryingadeviceserial
number...).
209
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Fileoperationsspecifictoeachopenfile!
Usingthepossibilitytoredefinefileoperationsforeachopenfile.
Process1
getPAL
video
Process2
ioctl:change
readfop
read
Openfile1
open
getNTSC
video!
read
Openfile2
open
/dev/video
2differentprocesses
canreaddifferentdata
fromthesamedevicefile!
210
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Fileoperations(4)
int(*mmap)(structfile*,
structvm_area_struct*);
Askingfordevicememorytobemapped
intotheaddressspaceofauserprocess.
Moreinourmmapsection.
Thesewerejustthemainones:
about25fileoperationscanbeset,correspondingtoall
thesystemcallsthatcanbeperformedonopenfiles.
211
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
readoperationexample
staticssize_t
acme_read(structfile*file,char__user*buf,size_tcount,loff_t*ppos)
{
/*Theacme_bufaddresscorrespondstoadeviceI/Omemoryarea*/
/*ofsizeacme_bufsize,obtainedwithioremap()*/
intremaining_size,transfer_size;
remaining_size=acme_bufsize(int)(*ppos);//byteslefttotransfer
if(remaining_size==0){/*Allread,returning0(EndOfFile)*/
return0;
}
/*Sizeofthistransfer*/
transfer_size=min(remaining_size,(int)count);
if(copy_to_user(buf/*to*/,acme_buf+*ppos/*from*/,transfer_size)){
returnEFAULT;
}else{/*Increasethepositionintheopenfile*/
*ppos+=transfer_size;
returntransfer_size;
}
}
Readmethod
Pieceofcodeavailablein
http://freeelectrons.com/doc/c/acme.c
212
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
writeoperationexample
staticssize_t
acme_write(structfile*file,constchar__user*buf,size_tcount,loff_t*ppos)
{
intremaining_bytes;
/*Numberofbytesnotwrittenyetinthedevice*/
remaining_bytes=acme_bufsize(*ppos);
if(count>remaining_bytes){
/*Can'twritebeyondtheendofthedevice*/
returnEIO;
}
if(copy_from_user(acme_buf+*ppos/*to*/,buf/*from*/,count)){
returnEFAULT;
}else{
/*Increasethepositionintheopenfile*/
*ppos+=count;
returncount;
}
}
Writemethod
Pieceofcodeavailablein
http://freeelectrons.com/doc/c/acme.c
213
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
fileoperationsdefinitionexample(3)
Definingafile_operationsstructure:
#include<linux/fs.h>
staticstructfile_operationsacme_fops=
{
.owner=THIS_MODULE,
.read=acme_read,
.write=acme_write,
};
Youjustneedtosupplythefunctionsyouimplemented!Defaultsfor
otherfunctions(suchasopen,release...)arefineifyoudonot
implementanythingspecial.
214
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Characterdeviceregistration(1)
Thekernelrepresentscharacterdriverswithacdevstructure
Declarethisstructureglobally(withinyourmodule):
#include<linux/cdev.h>
staticstructcdevacme_cdev;
Intheinitfunction,initializethestructure:
cdev_init(&acme_cdev,&acme_fops);
215
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Characterdeviceregistration(2)
Then,nowthatyourstructureisready,addittothesystem:
intcdev_add(
structcdev*p,
/*Characterdevicestructure*/
dev_tdev,
/*Startingdevicemajor/minornumber
*/
unsignedcount); /*Numberofdevices*/
Example(continued):
if(cdev_add(&acme_cdev,acme_dev,acme_count)){
printk(KERN_ERRChardriverregistrationfailed\n);
...
216
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Characterdeviceunregistration
Firstdeleteyourcharacterdevice:
voidcdev_del(structcdev*p);
Then,andonlythen,freethedevicenumber:
voidunregister_chrdev_region(dev_tfrom,
unsignedcount);
Example(continued):
cdev_del(&acme_cdev);
unregister_chrdev_region(acme_dev,acme_count);
217
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Linuxerrorcodes
Trytoreporterrorswitherrornumbersasaccurateas
possible!Fortunately,macronamesareexplicitandyou
canrememberthemquickly.
Genericerrorcodes:
include/asmgeneric/errnobase.h
Platformspecificerrorcodes:
include/asm/errno.h
218
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Chardriverexamplesummary(1)
staticvoid*acme_buf;
staticintacme_bufsize=8192;
staticintacme_count=1;
staticdev_tacme_dev;
staticstructcdevacme_cdev;
staticssize_tacme_write(...){...}
staticssize_tacme_read(...){...}
staticstructfile_operationsacme_fops=
{
.owner=THIS_MODULE,
.read=acme_read,
.write=acme_write
};
219
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Chardriverexamplesummary(2)
Showshowtohandleerrorsanddeallocateresourcesintherightorder!
staticint__initacme_init(void)
{
interr;
acme_buf=ioremap(ACME_PHYS,
acme_bufsize);
if(!acme_buf){
err=ENOMEM;
gotoerr_exit;
}
if(alloc_chrdev_region(&acme_dev,0,
acme_count,acme)){
err=ENODEV;
gotoerr_free_buf;
}
cdev_init(&acme_cdev,&acme_fops);
if(cdev_add(&acme_cdev,acme_dev,
acme_count)){
err=ENODEV;
gotoerr_dev_unregister;
}
return0;
err_dev_unregister:
unregister_chrdev_region(
acme_dev,acme_count);
err_free_buf:
iounmap(acme_buf);
err_exit:
returnerr;
}
staticvoid__exitacme_exit(void)
{
cdev_del(&acme_cdev);
unregister_chrdev_region(acme_dev,
acme_count);
iounmap(acme_buf);
}
Completeexamplecodeavailableonhttp://freeelectrons.com/doc/c/acme.c
220
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Systemadministration
Loadthecharacterdrivermodule
In/proc/devices,findthemajornumberituses.
Createthedevicefilewiththismajornumber
Thedevicefileisreadytouse!
Systemuser
Openthedevicefile,read,write,orsendioctl'stoit.
Kernel
Userspace
Definethefileoperationscallbacksforthedevicefile:read,write,ioctl...
Inthemoduleinitfunction,getmajorandminornumberswithalloc_chrdev_region(),
initacdevstructurewithyourfileoperationsandaddittothesystemwithcdev_add().
Inthemoduleexitfunction,callcdev_del()andunregister_chrdev_region()
Kernel
Characterdriverwriter
Kernel
Characterdriversummary
Executesthecorrespondingfileoperations
221
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
PracticallabCharacterdrivers
TimetostartLab5!
Writesimplefile_operations,for
acharacterdevice,includingioctl
controls
Registerthecharacterdevice
Usethekmallocandkfreeutilities
Exchangedatawithuserspace
222
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Driverdevelopment
Debugging
223
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Debuggingwithprintk
Universaldebuggingtechniqueusedsincethebeginningof
programming(firstfoundincavemendrawings)
Printedornotintheconsoleor/var/log/messages
accordingtothepriority.Thisiscontrolledbytheloglevel
kernelparameter,orthrough/proc/sys/kernel/printk
(seeDocumentation/sysctl/kernel.txt)
Availablepriorities(include/linux/kernel.h):
#defineKERN_EMERG"<0>"/*systemisunusable*/
#defineKERN_ALERT"<1>"/*actionmustbetakenimmediately*/
#defineKERN_CRIT"<2>"/*criticalconditions*/
#defineKERN_ERR"<3>"/*errorconditions*/
#defineKERN_WARNING"<4>"/*warningconditions*/
#defineKERN_NOTICE"<5>"/*normalbutsignificantcondition*/
#defineKERN_INFO"<6>"/*informational*/
#defineKERN_DEBUG"<7>"/*debuglevelmessages*/
224
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Debuggingwith/procor/sys(1)
Insteadofdumpingmessagesinthekernellog,youcanhaveyour
driversmakeinformationavailabletouserspace
Throughafilein/procor/sys,whichcontentsarehandledby
callbacksdefinedandregisteredbyyourdriver.
Canbeusedtoshowanypieceofinformation
aboutyourdeviceordriver.
Canalsobeusedtosenddatatothedriverortocontrolit.
Caution:anybodycanusethesefiles.
Youshouldremoveyourdebugginginterfaceinproduction!
Sincethearrivalofdebugfs,nolongerthepreferreddebugging
mechanism
225
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Debuggingwith/procor/sys(2)
Examples
cat/proc/acme/stats(dummyexample)
Displaysstatisticsaboutyouracmedriver.
cat/proc/acme/globals(dummyexample)
Displaysvaluesofglobalvariablesusedbyyourdriver.
echo600000>/sys/devices/system/cpu/cpu0/cpufreq/scaling_setspeed
AdjuststhespeedoftheCPU(controlledbythecpufreqdriver).
226
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Debugfs
Avirtualfilesystemtoexportdebugginginformationtouserspace.
Kernelconfiguration:DEBUG_FS
Kernelhacking>DebugFilesystem
Muchsimplertocodethananinterfacein/procor/sys.
ThedebugginginterfacedisappearswhenDebugfsisconfigured
out.
Youcanmountitasfollows:
sudomounttdebugfsnone/mnt/debugfs
Firstdescribedonhttp://lwn.net/Articles/115405/
APIdocumentedintheLinuxKernelFilesystemAPI:
http://freeelectrons.com/kerneldoc/latest/DocBook/filesystems/index.htm
227
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Simpledebugfsexample
#include<linux/debugfs.h>
staticchar*acme_buf;
staticunsignedlongacme_bufsize;
staticstructdebugfs_blob_wrapperacme_blob;
staticstructdentry*acme_buf_dentry;
//modulebuffer
staticu32acme_state;
staticstructdentry*acme_state_dentry;
//modulevariable
/*Moduleinit*/
acme_blob.data=acme_buf;
acme_blob.size=acme_bufsize;
acme_buf_dentry=debugfs_create_blob("acme_buf",S_IRUGO,
NULL,&acme_blob);
acme_state_dentry=debugfs_create_bool("acme_state",S_IRUGO,
NULL,&acme_state);
/*Moduleexit*/
debugfs_remove(acme_buf_dentry);
debugfs_remove(acme_state_dentry);
//Create
//newfiles
//indebugfs
//removingthefilesfromdebugfs
228
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Debuggingwithioctl
Canusetheioctl()systemcalltoqueryinformation
aboutyourdriver(ordevice)orsendcommandstoit.
Thiscallstheioctlfileoperationthatyoucanregisterin
yourdriver.
Advantage:yourdebugginginterfaceisnotpublic.
Youcouldevenleaveitwhenyoursystem(oritsdriver)isin
thehandsofitsusers.
229
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Debuggingwithgdb
Ifyouexecutethekernelfromadebuggeronthesame
machine,thiswillinterferewiththekernelbehavior.
However,youcanaccessthecurrentkernelstatewithgdb:
gdb/usr/src/linux/vmlinux/proc/kcore
uncompressedkernelkerneladdressspace
Youcanaccesskernelstructures,followpointers...
(readonly!)
Requiresthekerneltobecompiledwith
CONFIG_DEBUG_INFO(Kernelhackingsection)
230
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
kgdbAkerneldebugger
Theexecutionofthepatchedkernelisfullycontrolledby
gdbfromanothermachine,connectedthroughaserialline.
Candoalmosteverything,includinginsertingbreakpointsin
interrupthandlers.
IncludedinstandardLinuxsince2.6.26(x86andsparc).
arm,mipsandppcsupportmergedin2.6.27.
231
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Usingkgdb
Detailsavailableinthekerneldocumentation:
http://freeelectrons.com/kerneldoc/latest/DocBook/kgdb/
RecommendedtoturnonCONFIG_FRAME_POINTERtoaid
inproducingmorereliablestackbacktracesingdb.
YoumustincludeakgdbI/Odriver.Oneofthemiskgdb
overserialconsole(kgdboc:kgdboverconsole,enabled
byCONFIG_KGDB_SERIAL_CONSOLE)
Configurekgdbocatboottimebypassingtothekernel:
kgdboc=<ttydevice>,[baud].Forexample:
kgdboc=ttyS0,115200
232
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Usingkgdb(2)
Thenalsopasskgdbwaittothekernel:
itmakeskgdbwaitforadebuggerconnection.
Bootyourkernel,andwhentheconsoleisinitialized,
interruptthekernelwith[Alt][SyrRq][g].
Onyourworkstation,startgdbasfollows:
%gdb./vmlinux
(gdb)setremotebaud115200
(gdb)targetremote/dev/ttyS0
Onceconnected,youcandebugakernelthewayyouwould
debuganapplicationprogram.
233
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Kernelcrashanalysiswithkexec/kdump
kexecsystemcall:makesitpossibleto
callanewkernel,withoutrebootingand
goingthroughtheBIOS/firmware.
Idea:afterakernelpanic,makethe
kernelautomaticallyexecuteanew,
cleankernelfromareservedlocationin
RAM,toperformpostmortemanalysis
ofthememoryofthecrashedkernel.
1.Copydebug
kernelto
reserved
RAM
3.Analyze
crashed
kernelRAM
Standardkernel
2.kernel
panic,kexec
debugkernel
Debugkernel
SeeDocumentation/kdump/kdump.txt
inthekernelsourcesfordetails.
RegularRAM
234
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
DebuggingwithSystemTap
http://sourceware.org/systemtap/
Infrastructuretoaddinstrumentationtoarunningkernel:
tracefunctions,readandwritevariables,followpointers,gatherstatistics...
Eliminatestheneedtomodifythekernelsourcestoaddone'sown
instrumentationtoinvestigatedafunctionalorperformanceproblem.
Usesasimplescriptinglanguage.
Severalexamplescriptsandprobepointsareavailable.
BasedontheKprobesinstrumentationinfrastructure.
SeeDocumentation/kprobes.txtinkernelsources.
Linux2.6.26:supportedonmostpopularCPUs(armincludedin2.6.25).
However,lackofrecentsupportformips(2.6.16only!).
235
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
SystemTapscriptexample(1)
#!/usr/bin/envstap
#Usingstatisticsandmapstoexaminekernelmemoryallocations
globalkmalloc
probekernel.function("__kmalloc"){
kmalloc[execname()]<<<$size
}
#Exitafter10seconds
probetimer.ms(10000){exit()}
probeend{
foreach([name]inkmalloc){
printf("Allocationsfor%s\n",name)
printf("Count:%dallocations\n",@count(kmalloc[name]))
printf("Sum:%dKbytes\n",@sum(kmalloc[name])/1000)
printf("Average:%dbytes\n",@avg(kmalloc[name]))
printf("Min:%dbytes\n",@min(kmalloc[name]))
printf("Max:%dbytes\n",@max(kmalloc[name]))
print("\nAllocationsbysizeinbytes\n")
print(@hist_log(kmalloc[name]))
printf("\n\n");
}
}
236
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
SystemTapscriptexample(2)
#!/usr/bin/envstap
#Logseachfilereadperformedbyeachprocess
probekernel.function("vfs_read")
{
dev_nr=$file>f_dentry>d_inode>i_sb>s_dev
inode_nr=$file>f_dentry>d_inode>i_ino
printf("%s(%d)%s0x%x/%d\n",
execname(),pid(),probefunc(),dev_nr,inode_nr)
}
Nicetutorialonhttp://sources.redhat.com/systemtap/tutorial.pdf
237
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
UsingMagicSysRq
Linuxalsohas3fingerkeystosaveyourwork;)
Allowstorunmultipledebug/rescuecommandsevenwhenthe
kernelseemstobeindeeptrouble.Examplecommands:
[ALT][SysRq][d]:killsallprocesses,exceptinit.
[ALT][SysRq][n]:makesRTprocessesniceable.
[ALT][SysRq][s]:attemptstosyncallmountedfilesystems.
[ALT][SysRq][b]:immediatelyrebootwithoutsyncingand
unmounting.
Typicalcombination:[ALT][SysRq][s]
andthen[ALT][SysRq][b]
DetailedinDocumentation/sysrq.txt
238
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
DebuggingwithaJTAGinterface
FortheFOSScommunity,bestdonewithOpenOCD
(OpenOnChipDebugger):http://openocd.berlios.de/web/
OpenOCDsupportsmultipleJTAGhardwaredongles.
OpenOCDcaneitherbeinterfacedbyaserialconsole,adebugger,orbyan
Eclipsebasedinterface.
Ofcourse,itsupportsallthecapabilitiesofferedbyaJTAGinterface.
Seetheverycompletedocumentation:http://openocd.berlios.de/doc/
Foreachboard,you'llneedanOpenOCDconfigurationfile(askyoursupplier)
SeeveryusefuldetailsonusingEclipse/gcc/gdb/OpenOCDonWindows:
http://www2.amontec.com/sdk4arm/ext/jlynchtutorial20061124.pdfand
http://www.yagarto.de/howto/yagarto2/
239
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Kernelmarkers
Capabilitytoaddstaticmarkerstokernelcode,
mergedinLinux2.6.24byMatthieuDesnoyers.
Almostnoimpactonperformance,untilthemarkeris
dynamicallyenabled,byinsertingaprobekernelmodule.
Usefultoinserttracepointsthatwon'tbeimpactedby
changesintheLinuxkernelsources.
Seemarkerandprobeexample
insamples/markersinthekernelsources.
Seehttp://en.wikipedia.org/wiki/Kernel_marker
240
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
LTTng
http://ltt.polymtl.ca/
ThesuccessoroftheLinuxTraceToolkit(LTT)
Toolkitallowingtocollectandanalyzetracinginformation
fromthekernel,basedonkernelmarkersandkernel
tracepoints.
Sofar,basedonkernelpatches,butdoingitsbesttousein
treesolutions,andtobemergedinthefuture.
Veryprecisetimestamps,verylittleoverhead.
Usefulguidelinesin
http://ltt.polymtl.ca/svn/trunk/lttv/QUICKSTART
241
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
LTTV
ViewerforLTTngtraces
Supportforhugetraces(testedwith15GBones)
Cancombinemultipletracefilesinasingleview.
Graphicalortextinterface
Seehttp://ltt.polymtl.ca/lttvdoc/user_guide/
242
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Morekerneldebuggingtips
EnableCONFIG_KALLSYMS_ALL
(GeneralSetup>Configurestandardkernelfeatures)
togetoopsmessageswithsymbolnamesinsteadofrawaddresses
(thisobsoletestheksymoopstool).
Ifyourkerneldoesn'tbootyetorhangswithoutanymessage,youcan
activateLowLeveldebugging(KernelHackingsection,onlyavailableon
arm):
CONFIG_DEBUG_LL=y
TechniquestolocatetheCinstructionwhichcausedanoops:
http://kerneltrap.org/node/3648
Moreaboutkerneldebugginginthefree
LinuxDeviceDriversbook(Referencessection)!
243
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
PracticallabKerneldebugging
TimetostartLab6!
Loadabrokendriverandseeitcrash
Analyzetheerrorinformation
dumpedbythekernel.
Disassemblethecodeandlocate
theexactCinstructionwhichcaused
thefailure.
244
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxDriverDevelopment
Driverdevelopment
Processesandscheduling
245
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Processes
Aprocessisaninstanceofarunningprogram
Multipleinstancesofthesameprogramcanberunning.
Programcode(textsection)memoryisshared.
Eachprocesshasitsowndatasection,addressspace,processor
state,openfilesandpendingsignals.
Thekernelhasaseparatedatastructureforeachprocess.
246
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Threads
InLinux,threadsarejustimplementedasprocesses!
Newthreadsareimplementedasregularprocesses,
withtheparticularitythattheyarecreatedwiththesameaddress
space,filesystemresources,filedescriptorsandsignalhandlers
astheirparentprocess.
247
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Aprocesslife
EXIT_ZOMBIE
Parentprocess
Callsfork()
andcreates
anewprocess
Theprocessiselected
bythescheduler
TASK_RUNNING
Readybut
notrunning
Theeventoccurs
ortheprocessreceives
asignal.Processbecomes
runnableagain
Theprocessispreempted
bytoschedulertorun
ahigherprioritytask
TASK_INTERRUPTIBLE
TASK_UNINTERRUPTIBLE
orTASK_KILLABLE
Taskterminatedbutits
resourcesarenotfreedyet.
Waitingforitsparent
toacknowledgeitsdeath.
TASK_RUNNING
Actuallyrunning
Decidestosleep
onawaitqueue
foraspecificevent
Waiting
248
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Processcontext
Userspaceprogramsandsystemcallsarescheduledtogether
Processcontinuinginuserspace...
(orreplacedbyahigherpriorityprocess)
(canbepreempted)
Processexecutinginuserspace...
(canbepreempted)
Systemcall
orexception
Kernelcodeexecuted
onbehalfofuserspace
(canbepreemptedtoo!)
Stillhasaccesstoprocess
data(openfiles...)
249
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Kernelthreads
Thekerneldoesnotonlyreactfromuserspace(systemcalls,
exceptions)orhardwareevents(interrupts).Italsorunsitsown
processes.
Kernelthreadsarestandardprocessesscheduledandpreemptedin
thesameway(youcanviewthemwithtoporps!)Theyjusthave
nospecialaddressspaceandusuallyrunforever.
Kernelthreadexamples:
pdflush:regularlyflushesdirtymemorypagestodisk
(filechangesnotcommittedtodiskyet).
migration/<n>:PerCPUthreadstomigrateprocesses
betweenprocessors,tobalanceCPUloadbetweenprocessors.
250
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Processpriorities
Regularprocesses
Prioritiesfrom20(maximum)to19(minimum)
Onlyrootcansetnegativepriorities
(rootcangiveanegativeprioritytoaregularuserprocess)
Usethenicecommandtorunajobwithagivenpriority:
nicen<priority><command>
Usetherenicecommandtochangeaprocesspriority:
renice<priority>p<pid>
251
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Realtimepriorities
Processeswithrealtimeprioritycanbestartedbyrootusingthe
POSIXAPI
Availablethrough<sched.h>(seemansched.hfordetails)
100realtimeprioritiesavailable
SCHED_FIFOschedulingclass:
TheprocessrunsuntilcompletionunlessitisblockedbyanI/O,
voluntarilyrelinquishestheCPU,orispreemptedbyahigherpriority
process.
SCHED_RRschedulingclass:
Difference:theprocessesarescheduledinaRoundRobinway.
Eachprocessisrununtilitexhaustsamaxtimequantum.Thenother
processeswiththesamepriorityarerun,andsoandso...
252
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Timeslices
Theschedulerprioritizeshighpriorityprocesses
bygivingthemabiggertimeslice.
Initialprocesstimeslice:parent'stimeslicesplitin2
(otherwiseprocesswouldcheatbyforking).
Minimumpriority:5msor1jiffy(whicheverislarger)
Defaultpriorityinjiffies:100ms
Maximumpriority:800ms
Note:actuallydependsonHZ.
Seekernel/sched.cfordetails.
253
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Whenisschedulingrun?
Eachprocesshasaneed_reschedflagwhichisset:
Afteraprocessexhausteditstimeslice.
Afteraprocesswithahigherpriorityisawakened.
Thisflagischecked(possiblycausingtheexecutionofthe
scheduler)
Whenreturningtouserspacefromasystemcall
Whenreturningfrominterrupts(includingthecputimer),
whenkernelpreemptionisenabled.
Schedulingalsohappenswhenkernelcodeexplicitlyruns
schedule()orexecutesanactionthatsleeps.
254
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Driverdevelopment
Sleeping
255
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Sleeping
Sleepingisneededwhenaprocess(userspaceorkernelspace)
iswaitingfordata.
Userspaceprocess...
...Userspace
Other
processes
are
scheduled
readdevicefile
return
Systemcall...
askfor
data
sleep
...Systemcall
wakeup
Interrupt
handler
datareadynotification
256
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Howtosleep(1)
Mustdeclareawaitqueue
Staticqueuedeclaration
DECLARE_WAIT_QUEUE_HEAD(module_queue);
Ordynamicqueuedeclaration
wait_queue_head_tqueue;
init_waitqueue_head(&queue);
257
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Howtosleep(2)
Severalwaystomakeakernelprocesssleep
wait_event(queue,condition);
SleepsuntilthegivenCexpressionistrue.
Caution:can'tbeinterrupted(can'tkilltheuserspaceprocess!)
wait_event_killable(queue,condition);(SinceLinux2.6.25)
SleepsuntilthegivenCexpressionistrue.
Canonlybeinterruptedbyafatalsignal(SIGKILL)
wait_event_interruptible(queue,condition);
Canbeinterruptedbyanysignal
wait_event_timeout(queue,condition,timeout);
Sleepsandautomaticallywakesupafterthegiventimeout.
wait_event_interruptible_timeout(queue,condition,timeout);
Sameasabove,interruptible.
258
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
HowtosleepExample
Fromdrivers/ieee1394/video1394.c
wait_event_interruptible(
d>waitq,
(d>buffer_status[v.buffer]
==VIDEO1394_BUFFER_READY)
);
if(signal_pending(current))
returnEINTR;
Currentlyrunningprocess
259
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Wakingup!
Typicallydonebyinterrupthandlerswhendatasleeping
processesarewaitingforareavailable.
wake_up(queue);
Wakesupallthewaitingprocessesonthegivenqueue
wake_up_interruptible(queue);
Wakesuponlytheprocesseswaitinginaninterruptible
sleeponthegivenqueue
260
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Sleepingandwakingupimplementation
Theschedulerdoesn'tkeepevaluatingthesleepingcondition!
wait_event_interruptible(queue,condition);
TheprocessisputintheTASK_INTERRUPTIBLEstate.
wake_up_interruptible(queue);
Forallprocesseswaitinginqueue,conditionisevaluated.
Whenitevaluatestotrue,theprocessisputback
totheTASK_RUNNINGstate,andtheneed_reschedflagforthe
currentprocessisset.
Thisway,severalprocessescanbewokenupatthesametime.
261
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Driverdevelopment
Interruptmanagement
262
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Interrupthandlerconstraints
Notrunfromausercontext:
Can'ttransferdatatoandfromuserspace
(needtobedonebysystemcallhandlers)
InterrupthandlerexecutionismanagedbytheCPU,not
bythescheduler.Handlerscan'trunactionsthatmay
sleep,becausethereisnothingtoresumetheir
execution.Inparticular,needtoallocatememorywith
GFP_ATOMIC.
Havetocompletetheirjobquicklyenough:
theyshouldn'tblocktheirinterruptlinefortoolong.
263
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Registeringaninterrupthandler(1)
Definedininclude/linux/interrupt.h
intrequest_irq(
Returns0ifsuccessful
unsignedintirq,
Requestedirqchannel
irqreturn_thandler,
Interrupthandler
unsignedlongirq_flags,
Optionmask(seenextpage)
constchar*devname,
Registeredname
void*dev_id);
Pointertosomehandlerdata
CannotbeNULLandmustbeuniqueforsharedirqs!
voidfree_irq(unsignedintirq,void*dev_id);
dev_idcannotbeNULLandmustbeuniqueforsharedirqs.
Otherwise,onasharedinterruptline,
free_irqwouldn'tknowwhichhandlertofree.
264
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Registeringaninterrupthandler(2)
irq_flagsbitvalues(canbecombined,noneisfinetoo)
IRQF_DISABLED
"Quick"interrupthandler.Runwithallinterruptsdisabledonthecurrentcpu
(insteadofjustthecurrentline).Forlatencyreasons,shouldonlybeused
whenneeded!
IRQF_SHARED
Runwithinterruptsdisabledonlyonthecurrentirqlineandonthelocalcpu.
Theinterruptchannelcanbesharedbyseveraldevices.Requiresa
hardwarestatusregistertellingwhetheranIRQwasraisedornot.
IRQF_SAMPLE_RANDOM
Interruptscanbeusedtocontributetothesystementropypoolusedby
/dev/randomand/dev/urandom.Usefultogenerategoodrandom
numbers.Don'tusethisiftheinterruptbehaviorofyourdeviceis
predictable!
265
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Whentoregisterthehandler
Eitheratdriverinitializationtime:
consumeslotsofIRQchannels!
Oratdeviceopentime(firstcalltotheopenfileoperation):
betterforsavingfreeIRQchannels.
Needtocountthenumberoftimesthedeviceisopened,to
beabletofreetheIRQchannelwhenthedeviceisno
longerinuse.
266
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Informationoninstalledhandlers
/proc/interrupts
CPU0
0:5616905XTPICtimer#Registeredname
1:9828XTPICi8042
2:0XTPICcascade
3:1014243XTPICorinoco_cs
7:184XTPICIntel82801DBICH4
8:1XTPICrtc
9:2XTPICacpi
11:566583XTPICehci_hcd,uhci_hcd,
uhci_hcd,uhci_hcd,yenta,yenta,radeon@PCI:1:0:0
12:5466XTPICi8042
14:121043XTPICide0
15:200888XTPICide1
NMI:0
NonMaskableInterrupts
ERR:0
Spuriousinterruptcount
267
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Totalnumberofinterrupts
cat/proc/stat|grepintr
intr819076760929671037701102775520196...
Totalnumber
ofinterrupts
IRQ1
total
IRQ2 IRQ3
total
...
268
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Theinterrupthandler'sjob
Acknowledgetheinterrupttothedevice
(otherwisenomoreinterruptswillbegenerated)
Read/writedatafrom/tothedevice
Wakeupanywaitingprocesswaitingforthecompletion
ofthisread/writeoperation:
wake_up_interruptible(&module_queue);
269
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Interrupthandlerprototype
irqreturn_t(*handler)(
int, //irqnumberofthecurrentinterrupt
void*dev_id, //Pointerusedtokeeptrack
//ofthecorrespondingdevice.
//Usefulwhenseveraldevices
//aremanagedbythesamemodule
);
Returnvalue:
IRQ_HANDLED:recognizedandhandledinterrupt
IRQ_NONE:notonadevicemanagedbythemodule.Usefulto
shareinterruptchannelsand/orreportspuriousinterruptsto
thekernel.
270
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Tophalfandbottomhalfprocessing(1)
Splittingtheexecutionofinterrupthandlersin2parts
Tophalf:theinterrupthandlermustcompleteasquickly
aspossible.Onceitacknowledgedtheinterrupt,itjust
schedulesthelengthyrestofthejobtakingcareofthe
data,foralaterexecution.
Bottomhalf:completingtherestoftheinterrupthandler
job.Handlesdata,andthenwakesupanywaitinguser
process.
Bestimplementedbytasklets(alsocalledsoftirqs).
271
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
tophalfandbottomhalfprocessing(2)
Declarethetaskletinthemodulesourcefile:
DECLARE_TASKLET(module_tasklet,/*name*/
module_do_tasklet,/*function*/
data/*params*/
);
Schedulethetaskletinthetophalfpart(interrupthandler):
tasklet_schedule(&module_tasklet);
Notethatatasklet_hi_schedulefunctionisavailableto
definehighprioritytaskletstorunbeforeordinaryones.
Bydefault,taskletsareexecutedrightafteralltophalves(hard
irqs)
272
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Disablinginterrupts
Maybeusefulinregulardrivercode...
Canbeusefultoensurethataninterrupthandlerwillnotpreemptyourcode
(includingkernelpreemption)
DisablinginterruptsonthelocalCPU:
unsignedlongflags;
local_irq_save(flags);
//Interruptsdisabled
...
local_irq_restore(flags);//Interruptsrestoredtotheirpreviousstate.
Note:mustberunfromwithinthesamefunction!
273
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Maskingoutaninterruptline
Usefultodisableinterruptsonaparticularline
voiddisable_irq(unsignedintirq);
Disablestheirqlineforallprocessorsinthesystem.
Waitsforallcurrentlyexecutinghandlerstocomplete.
voiddisable_irq_nosync(unsignedintirq);
Same,exceptitdoesn'twaitforhandlerstocomplete.
voidenable_irq(unsignedintirq);
Restoresinterruptsontheirqline.
voidsynchronize_irq(unsignedintirq);
Waitsforirqhandlerstocomplete(ifany).
274
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Checkinginterruptstatus
Canbeusefulforcodewhichcanberunfrombothprocessor
interruptcontext,toknowwhetheritisallowedornottocallcode
thatmaysleep.
irqs_disabled()
Testswhetherlocalinterruptdeliveryisdisabled.
in_interrupt()
Testswhethercodeisrunningininterruptcontext
in_irq()
Testswhethercodeisrunninginaninterrupthandler.
275
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Interruptmanagementfun
Inatraininglab,somebodyforgottounregisterahandleron
asharedinterruptlineinthemoduleexitfunction.
Whydidhiskernelcrashwithasegmentationfault
atmoduleunload?
Answer...
Inatraininglab,somebodyfreedthetimerinterrupthandler
bymistake(usingthewrongirqnumber).Thesystemfroze.
Rememberthekernelisnotprotectedagainstitself!
276
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Interruptmanagementsummary
Devicedriver
Whenthedevicefileisfirst
open,registeraninterrupt
handlerforthedevice's
interruptchannel.
Interrupthandler
Calledwhenaninterruptis
raised.
Acknowledgetheinterrupt
Tasklet
Processthedata
Wakeupprocesseswaiting
forthedata
Devicedriver
Whenthedeviceisnolonger
openedbyanyprocess,
unregistertheinterrupt
handler.
Ifneeded,scheduleatasklet
takingcareofhandlingdata.
Otherwise,wakeupprocesses
waitingforthedata.
277
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
PracticallabInterrupts
TimetostartLab7!
Implementasimpleinterrupthandler
Registerthishandleronashared
interruptlineonyourGNU/Linuxhost.
SeehowLinuxhandles
sharedinterruptlines.
278
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Driverdevelopment
Concurrentaccesstoresources
279
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Sourcesofconcurrencyissues
Thesameresourcescanbeaccessedbyseveralkernel
processesinparallel,causingpotentialconcurrencyissues
Severaluserspaceprogramsaccessingthesamedevicedata
orhardware.Severalkernelprocessescouldexecutethesame
codeonbehalfofuserprocessesrunninginparallel.
Multiprocessing:thesamedrivercodecanberunningon
anotherprocessor.ThiscanalsohappenwithsingleCPUswith
hyperthreading.
Kernelpreemption,interrupts:kernelcodecanbeinterruptedat
anytime(justafewexceptions),andthesamedatamaybe
accessbyanotherprocessbeforetheexecutioncontinues.
280
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Avoidingconcurrencyissues
Avoidusingglobalvariablesandshareddatawhenever
possible
(cannotbedonewithhardwareresources).
Usetechniquestomanageconcurrentaccessto
resources.
SeeRustyRussell'sUnreliableGuideToLocking
Documentation/DocBook/kernellocking/
inthekernelsources.
281
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Concurrencyprotectionwithlocks
Process1
Process2
Failed
Acquirelock
Success
Criticalcodesection
Tryagain
Waitlockrelease
Success
Sharedresource
Releaselock
282
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Linuxmutexes
ThemainlockingprimitivesinceLinux2.6.16.
Betterthancountingsemaphoreswhenbinaryonesare
enough.
Mutexdefinition:
#include<linux/mutex.h>
Initializingamutexstatically:
DEFINE_MUTEX(name);
Orinitializingamutexdynamically:
voidmutex_init(structmutex*lock);
283
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
lockingandunlockingmutexes
voidmutex_lock(structmutex*lock);
Triestolockthemutex,sleepsotherwise.
Caution:can'tbeinterrupted,resultinginprocessesyoucannotkill!
intmutex_lock_killable(structmutex*lock);
Same,butcanbeinterruptedbyafatal(SIGKILL)signal.Ifinterrupted,returns
anonzerovalueanddoesn'tholdthelock.Testthereturnvalue!!!
intmutex_lock_interruptible(structmutex*lock);
Same,butcanbeinterruptedbyanysignal.
intmutex_trylock(structmutex*lock);
Neverwaits.Returnsanonzerovalueifthemutexisnotavailable.
intmutex_is_locked(structmutex*lock);
Justtellswhetherthemutexislockedornot.
voidmutex_unlock(structmutex*lock);
Releasesthelock.Doitassoonasyouleavethecriticalsection.
284
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Reader/writersemaphores
Allowsharedaccessbyunlimitedreaders,orbyonly1writer.Writersget
priority.
voidinit_rwsem(structrw_semaphore*sem);
voiddown_read(structrw_semaphore*sem);
intdown_read_trylock(structrw_semaphore*sem);
intup_read(structrw_semaphore*sem);
voiddown_write(structrw_semaphore*sem);
intdown_write_trylock(structrw_semaphore*sem);
intup_write(structrw_semaphore*sem);
Wellsuitedforrarewrites,holdingthesemaphorebriefly.Otherwise,
readersgetstarved,waitingtoolongforthesemaphoretobereleased.
285
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Spinlocks
Lockstobeusedforcodethatisnotallowedtosleep
(interrupthandlers),orthatdoesn'twanttosleep(critical
sections).Beverycarefulnottocallfunctionswhichcan
sleep!
Originallyintendedformultiprocessorsystems
Spinlocksneversleepandkeepspinning
inaloopuntilthelockisavailable.
Stilllocked?
Spinlock
Spinlockscausekernelpreemptiontobedisabled
ontheCPUexecutingthem.
286
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Initializingspinlocks
Static
spinlock_tmy_lock=SPIN_LOCK_UNLOCKED;
Dynamic
voidspin_lock_init(spinlock_t*lock);
287
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Usingspinlocks(1)
Severalvariants,dependingonwherethespinlockiscalled:
voidspin_[un]lock(spinlock_t*lock);
Doesn'tdisableinterrupts.Usedforlockinginprocesscontext
(criticalsectionsinwhichyoudonotwanttosleep).
voidspin_lock_irqsave/spin_unlock_irqrestore
(spinlock_t*lock,unsignedlongflags);
Disables/restoresIRQsonthelocalCPU.
Typicallyusedwhenthelockcanbeaccessedinbothprocess
andinterruptcontext,topreventpreemptionbyinterrupts.
288
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Usingspinlocks(2)
voidspin_[un]lock_bh(spinlock_t*lock);
Disablessoftwareinterrupts,butnothardwareones.
Usefultoprotectshareddataaccessedinprocesscontext
andinasoftinterrupt(bottomhalf).Noneedtodisable
hardwareinterruptsinthiscase.
Notethatreader/writerspinlocksalsoexist.
289
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Deadlocksituations
Theycanlockupyoursystem.Makesuretheyneverhappen!
Don'tcallafunctionthatcan
trytogetaccesstothesame
lock
Holdingmultiplelocksisrisky!
Getlock1
Getlock1
Getlock2
call
Getlock2
Waitforlock1
Dead
Lock!
Getlock1
Dead
Lock!
290
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Kernellockvalidator
FromIngoMolnarandArjanvandeVen
Addsinstrumentationtokernellockingcode
Detectviolationsoflockingrulesduringsystemlife,suchas:
Locksacquiredindifferentorder
(keepstrackoflockingsequencesandcomparesthem).
Spinlocksacquiredininterrupthandlersandalsoinprocess
contextwheninterruptsareenabled.
Notsuitableforproductionsystemsbutacceptableoverheadin
development.
SeeDocumentation/lockdepdesign.txtfordetails
291
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Alternativestolocking
Aswehavejustseen,lockingcanhaveastrongnegative
impactonsystemperformance.Insomesituations,youcould
dowithoutit.
ByusinglockfreealgorithmslikeReadCopyUpdate(RCU).
RCUAPIavailableinthekernel
(Seehttp://en.wikipedia.org/wiki/RCU).
Whenavailable,useatomicoperations.
292
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Atomicvariables
Usefulwhenthesharedresourceisan
integervalue
Evenaninstructionliken++isnot
guaranteedtobeatomiconall
processors!
Header
#include<asm/atomic.h>
Type
atomic_t
containsasignedinteger(atleast24
bits)
Operationswithoutreturnvalue:
voidatomic_inc(atomic_t*v);
voidatomic_dec(atomic_t*v);
voidatomic_add(inti,atomic_t*v);
voidatomic_sub(inti,atomic_t*v);
Simularfunctionstestingtheresult:
intatomic_inc_and_test(...);
intatomic_dec_and_test(...);
intatomic_sub_and_test(...);
Functionsreturningthenewvalue:
intatomic_inc_and_return(...);
intatomic_dec_and_return(...);
intatomic_add_and_return(...);
intatomic_sub_and_return(...);
Atomicoperations(mainones)
Setorreadthecounter:
atomic_set(atomic_t*v,inti);
intatomic_read(atomic_t*v);
293
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Atomicbitoperations
Supplyveryfast,atomicoperations
Onmostplatforms,applytoanunsignedlongtype.
Applytoavoidtypeonafewothers.
Set,clear,toggleagivenbit:
voidset_bit(intnr,unsignedlong*addr);
voidclear_bit(intnr,unsignedlong*addr);
voidchange_bit(intnr,unsignedlong*addr);
Testbitvalue:
inttest_bit(intnr,unsignedlong*addr);
Testandmodify(returnthepreviousvalue):
inttest_and_set_bit(...);
inttest_and_clear_bit(...);
inttest_and_change_bit(...);
294
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Driverdevelopment
mmap
295
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
mmap(1)
Possibilitytohavepartsofthevirtualaddressspaceofaprogram
mappedtothecontentsofafile!
>cat/proc/1/maps(initprocess)
startend
permoffsetmajor:minorinodemappedfilename
007710000077f000rxp0000000003:051165839/lib/libselinux.so.1
0077f00000781000rwp0000d00003:051165839/lib/libselinux.so.1
0097d00000992000rxp0000000003:051158767/lib/ld2.3.3.so
0099200000993000rp0001400003:051158767/lib/ld2.3.3.so
0099300000994000rwp0001500003:051158767/lib/ld2.3.3.so
0099600000aac000rxp0000000003:051158770/lib/tls/libc2.3.3.so
00aac00000aad000rp0011600003:051158770/lib/tls/libc2.3.3.so
00aad00000ab0000rwp0011700003:051158770/lib/tls/libc2.3.3.so
00ab000000ab2000rwp00ab000000:000
0804800008050000rxp0000000003:05571452/sbin/init(text)
0805000008051000rwp0000800003:05571452/sbin/init(data,stack)
08b4300008b64000rwp08b4300000:000
f6fdf000f6fe0000rwpf6fdf00000:000
fefd4000ff000000rwpfefd400000:000
ffffe000fffff000p0000000000:000
296
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
mmap(2)
Particularlyusefulwhenthefileisadevicefile!
AllowstoaccessdeviceI/Omemoryandportswithouthavingto
gothrough(expensive)read,writeorioctlcalls!
Xserverexample(mapsexcerpt)
startend
permoffsetmajor:minorinodemappedfilename
08047000081be000rxp0000000003:05310295/usr/X11R6/bin/Xorg
081be000081f0000rwp0017600003:05310295/usr/X11R6/bin/Xorg
...
f4e08000f4f09000rwse000000003:05655295/dev/dri/card0
f4f09000f4f0b000rws4281a00003:05655295/dev/dri/card0
f4f0b000f6f0b000rwse800000003:05652822/dev/mem
f6f0b000f6f8b000rwsfcff000003:05652822/dev/mem
Amoreuserfriendlywaytogetsuchinformation:pmap<pid>
297
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
mmapoverview
Process
mmap
system
call(once)
access
virtual
address
Processvirtualaddressspace
MMU
Devicedriver
mmapfopcalled
initializesthemapping
access
physical
address
Physicaladdressspace
298
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
HowtoimplementmmapUserspace
Openthedevicefile
Callthemmapsystemcall(seemanmmapfordetails):
void*mmap(
void*start,
/*Often0,preferredstartingaddress*/
size_tlength, /*Lengthofthemappedarea*/
intprot,
/*Permissions:read,write,execute*/
intflags,
/*Options:sharedmapping,privatecopy...
*/
intfd,
/*Openfiledescriptor*/
off_toffset
/*Offsetinthefile*/
);
Yougetavirtualaddressyoucanwritetoorreadfrom.
299
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
HowtoimplementmmapKernelspace
Characterdriver:implementammapfileoperation
andaddittothedriverfileoperations:
int(*mmap)(
structfile*,
/*Openfilestructure*/
structvm_area_struct* /*KernelVMAstructure*/
);
Initializethemapping.
Canbedoneinmostcaseswiththeremap_pfn_range()
function,whichtakescareofmostofthejob.
300
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
remap_pfn_range()
pfn:pageframenumber
Themostsignificantbitsofthepageaddress
(withoutthebitscorrespondingtothepagesize).
#include<linux/mm.h>
intremap_pfn_range(
structvm_area_struct*,
/*VMAstruct*/
unsignedlongvirt_addr, /*Startinguservirtualaddress*/
unsignedlongpfn, /*pfnofthestartingphysicaladdress*/
unsignedlongsize,
/*Mappingsize*/
pgprot_t
/*Pagepermissions*/
);
301
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Simplemmapimplementation
staticintacme_mmap(
structfile*file,structvm_area_struct*vma)
{
size=vma>vm_startvma>vm_end;
if(size>ACME_SIZE)
returnEINVAL;
if(remap_pfn_range(vma,
vma>vm_start,
ACME_PHYS>>PAGE_SHIFT,
size,
vma>vm_page_prot))
returnEAGAIN;
return0;
}
302
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
devmem2
http://freeelectrons.com/pub/mirror/devmem2.c,byJanDerkBakker
Veryusefultooltodirectlypeek(read)orpoke(write)I/Oaddresses
mappedinphysicaladdressspacefromashellcommandline!
Veryusefulforearlyinteractionexperimentswithadevice,without
havingtocodeandcompileadriver.
Usesmmapto/dev/mem.
Examples(b:byte,h:half,w:word)
devmem20x000c0004h(reading)
devmem20x000c0008w0xffffffff(writing)
303
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
mmapsummary
Thedevicedriverisloaded.
Itdefinesanmmapfileoperation.
Auserspaceprocesscallsthemmapsystemcall.
Themmapfileoperationiscalled.
Itinitializesthemappingusingthedevicephysicaladdress.
Theprocessgetsastartingaddresstoreadfromandwriteto
(dependingonpermissions).
TheMMUautomaticallytakescareofconvertingtheprocess
virtualaddressesintophysicalones.
Directaccesstothehardware!
Noexpensivereadorwritesystemcalls!
304
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Driverdevelopment
DeviceModel
305
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Unifieddevicemodel
The2.6kernelincludedasignificantnewfeature:
aunifieddevicemodel
Insteadofhavingdifferentadhocmechanismsinthe
varioussubsystems,thedevicemodelunifiesthe
descriptionofthedevicesandtheirtopology
Minimizationofcodeduplication
Commonfacilities(referencecounting,eventnotification,
powermanagement,etc.)
Enumeratethedevices,viewtheirinterconnections,linkthe
devicestotheirbusesanddrivers,categorizethembyclasses
306
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Busdrivers
Thefirstcomponentofthedevicemodelisthebusdriver
Onebusdriverforeachtypeofbus:USB,PCI,SPI,MMC,
ISA,etc.
Itisresponsiblefor
Registeringthebustype
Allowingregistrationofadapterdrivers(USBcontrollers,I2C
adapters,etc.),ableofdetectingtheconnecteddevices
Allowingregistrationofdevicedrivers(USBdevices,I2C
devices,PCIdevices,etc.),managingthedevices
Matchingthedevicedriversagainstthedevicesdetectedby
theadapterdrivers
307
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Listofdeviceidentifiers
Dependingonthebustype,themethodforbindingadevice
toadriverisdifferent.Formanybuses,itisbasedonunique
identifiers
Thedevicedriverdefinesatablewiththelistofdevice
identifiersitisabletomanage:
staticconststructpci_device_idrhine_pci_tbl[]={
{0x1106,0x3043,PCI_ANY_ID,PCI_ANY_ID,},/*VT86C100A*/
{0x1106,0x3053,PCI_ANY_ID,PCI_ANY_ID,},/*VT6105M*/
{}/*terminatelist*/
};
MODULE_DEVICE_TABLE(pci,rhine_pci_tbl);
Codeonthisslideandthenextslidesaretaken
fromtheviarhinedriverindrivers/net/viarhine.c
308
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Definingthedriver
Thedevicedriverdefinesadriverstructure,usually
specializedbythebusinfrastructure(pci_driver,
usb_driver,etc.)
Thestructurepointsto:thedevicetable,aprobefunction,
calledwhenadeviceisdetectedandvariousothercallbacks
staticstructpci_driverrhine_driver={
.name=DRV_NAME,
.id_table=rhine_pci_tbl,
.probe=rhine_init_one,
.remove=__devexit_p(rhine_remove_one),
#ifdefCONFIG_PM
.suspend=rhine_suspend,
.resume=rhine_resume,
#endif/*CONFIG_PM*/
.shutdown=rhine_shutdown,
};
309
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Registeringthedriver
Inthemoduleinitializationfunction,thedriverisregistered
tothebusinfrastructure,inordertoletthebusknowthatthe
driverisavailabletohandledevices.
staticint__initrhine_init(void)
{
[...]
returnpci_register_driver(&rhine_driver);
}
staticvoid__exitrhine_cleanup(void)
{
pci_unregister_driver(&rhine_driver);
}
IfanewPCIdevicematchesoneoftheidentifiersofthe
table,theprobe()methodofthePCIdriverwillgetcalled.
310
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Probemethod
Theprobe()methodreceivesasargumentastructure
describingthedevice,usuallyspecializedbythebus
infrastructure(pci_dev,usb_device,etc.)
Thisfunctionisresponsiblefor
Initializingthedevice,mappingtheI/Omemory,registeringthe
interrupthandlers.Thebusinfrastructureprovidesmethodsto
gettheaddresses,interruptsnumbersandotherdevice
specificinformation.
Registeringthedevicetotheproperkernelframework,for
examplethenetworkinfrastructure.
311
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Devicedriver(5)
staticint__devinitrhine_init_one(structpci_dev*pdev,
conststructpci_device_id*ent)
{
structnet_device*dev;
[...]
rc=pci_enable_device(pdev);
[...]
pioaddr=pci_resource_start(pdev,0);
memaddr=pci_resource_start(pdev,1);
[...]
dev=alloc_etherdev(sizeof(structrhine_private));
[...]
SET_NETDEV_DEV(dev,&pdev>dev);
[...]
rc=pci_request_regions(pdev,DRV_NAME);
[...]
ioaddr=pci_iomap(pdev,bar,io_size);
[...]
rc=register_netdev(dev);
[...]
}
312
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Globalarchitecture
PCIbusdriver
Thedevice
driver
registers
itselfandthe
device
identifiersto
thebus
driver
Thebusdriver
detectsa
matchingdevice,
andcallsthe
probe()method
ofthedevice
driver.
viarhine
devicedriver
Network
device
framework
3
Theprobe()
methodofthe
devicedriver
initializesthe
deviceand
registersanew
networkinterface
313
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
sysfs
Thebus,device,drivers,etc.structuresareinternaltothe
kernel
Thesysfsvirtualfilesystemprovidesamechanismtoexport
suchinformationtouserspace
Usedforexamplebyudevtoprovideautomaticmodule
loading,firmwareloading,devicefilecreation,etc.
sysfsisusuallymountedin/sys
/sys/bus/containsthelistofbuses
/sys/devices/containsthelistofdevices
/sys/classenumeratesdevicesbyclass(net,input,
block...),whateverthebustheyareconnectedto.Veryuseful!
Takeyourtimetoexplore/sysonyourworkstation.
314
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Platformdevices
Onembeddedsystems,devicesareoftennotconnected
throughabusallowingenumeration,hotplugging,and
providinguniqueidentifiersfordevices
However,westillwantthedevicestobepartofthedevice
model
Thesolutiontothisistheplatformdriver/platformdevice
infrastructure
Theplatformdevicesarethedevicesthataredirectly
connectedtotheCPU,withoutanykindofbus
315
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Implementationoftheplatformdriver
Thedriverimplementsaplatform_driverstructure
(exampletakenfromdrivers/serial/imx.c)
staticstructplatform_driverserial_imx_driver={
.probe=serial_imx_probe,
.remove=serial_imx_remove,
.driver={
.name="imxuart",
.owner=THIS_MODULE,
},
};
Andregistersitsdrivertotheplatformdriverinfrastructure
staticint__initimx_serial_init(void)
{
[...]
ret=platform_driver_register(&serial_imx_driver);
[...]
}
316
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Instantiationoftheplatformdevice
Intheboardspecificcode,theplatformdevicesare
instantiated(arch/arm/machimx/mx1ads.c):
staticstructplatform_deviceimx_uart1_device={
.name="imxuart",
.id=0,
.num_resources=ARRAY_SIZE(imx_uart1_resources),
.resource=imx_uart1_resources,
.dev={
.platform_data=&uart_pdata,
}
};
Thematchbetweenthedeviceandthedriverismadeusing
thename.Itmustbeuniqueamongstdrivers!
317
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Instantiationoftheplatformdevice(2)
Thedeviceispartofalist
staticstructplatform_device*devices[]__initdata={
&cs89x0_device,
&imx_uart1_device,
&imx_uart2_device,
};
Andthelistofdevicesisaddedtothesystem
duringboardinitialization
staticvoid__initmx1ads_init(void)
{
[...]
platform_add_devices(devices,ARRAY_SIZE(devices));
}
318
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
I/Oresources
EachplatformdeviceisassociatedwithasetofI/Oresources,
referencedintheplatform_devicestructure
staticstructresourceimx_uart1_resources[]={
[0]={
.start=0x00206000,
.end=0x002060FF,
.flags=IORESOURCE_MEM,
},
[1]={
.start=(UART1_MINT_RX),
.end=(UART1_MINT_RX),
.flags=IORESOURCE_IRQ,
},
};
ItallowsthedrivertobeindependentfortheI/Oaddresses,IRQ
numbers!Seeimx_uart2_deviceforanotherdeviceusingthe
sameplatformdriver.
319
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Insidetheplatformdriver
Whenaplatform_deviceisaddedtothesystemusing
platform_add_device(),theprobe()methodofthe
platformdrivergetscalled
Thismethodisresponsibleforinitializingthehardware,
registeringthedevicetotheproperframework(inourcase,
theserialdriverframework)
TheplatformdriverhasaccesstotheI/Oresources:
res=platform_get_resource(pdev,IORESOURCE_MEM,0);
base=ioremap(res>start,PAGE_SIZE);
sport>rxirq=platform_get_irq(pdev,0);
320
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
sysfstools
http://linuxdiag.sourceforge.net/Sysfsutils.html
libsysfsThelibrary'spurposeistoprovideaconsistent
andstableinterfaceforqueryingsystemdeviceinformation
exposedthroughsysfs.
systoolAutilitybuiltuponlibsysfsthatlistsdevices
bybus,class,andtopology.
321
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
References
Kerneldocumentation
Documentation/drivermodel/
Documentation/filesystems/sysfs.txt
Linux2.6DeviceModel
http://www.bravegnu.org/devicemodel/devicemodel.html
LinuxDeviceDrivers,chapter14TheLinuxDeviceModel
http://lwn.net/images/pdf/LDD3/ch14.pdf
Thekernelsourcecode
Fullofexamplesofotherdrivers!
322
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Adviceandresources
Gettinghelpandcontributions
323
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Solvingissues
Ifyoufaceanissue,anditdoesn'tlookspecifictoyourwork
butrathertothetoolsyouareusing,itisverylikelythat
someoneelsealreadyfacedit.
SearchtheInternetforsimilarerrorreports.
Youhavegreatchancesoffindingasolutionorworkaround,
oratleastanexplanationforyourissue.
Otherwise,reportingtheissueisuptoyou!
324
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Gettinghelp
Ifyouhaveasupportcontract,askyourvendor.
Otherwise,don'thesitatetoshareyourquestionsandissues
EithercontacttheLinuxmailinglistforyourarchitecture(likelinux
armkernelorlinuxshdev...).
Orcontactthemailinglistforthesubsystemyou'redealingwith
(linuxusbdevel,linuxmtd...).Don'taskthemaintainerdirectly!
MostmailinglistscomewithaFAQpage.Makesureyoureadit
beforecontactingthemailinglist.
UsefulIRCresourcesareavailabletoo
(forexampleonhttp://kernelnewbies.org).
RefrainfromcontactingtheLinuxKernelmailinglist,unlessyou're
anexperienceddeveloperandneedadvice.
325
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Gettingcontributions
Appliesifyourprojectcaninterestotherpeople:
developingadriverorfilesystem,portingLinuxonanew
processor,boardordeviceavailableonthemarket...
Externalcontributorscanhelpyoualotby
Testing
Writingdocumentation
Makingsuggestions
Evenwritingcode
326
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Encouragingcontributions
Openyourdevelopmentprocess:mailinglist,Wiki,public
CVSreadaccess
Leteveryonecontributeaccordingtotheirskillsand
interests.
Releaseearly,releaseoften
Takefeedbackandsuggestionsintoaccount
Recognizecontributions
Makesurestatusanddocumentationareuptodate
Publicizeyourworkandprogresstobroaderaudiences
327
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Adviceandresources
Bugreportandpatchsubmission
328
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
ReportingLinuxbugs
Firstmakesureyou'reusingthelatestversion
Makesureyouinvestigatetheissueasmuchasyoucan:
seeDocumentation/BUGHUNTING
Makesurethebughasnotbeenreportedyet.Abugtracking
system
(http://bugzilla.kernel.org/)existsbutveryfewkerneldevelopers
useit.Besttousewebsearchengines(accessingpublicmailing
listarchives)
Ifthesubsystemyoureportabugonhasamailinglist,useit.
Otherwise,contacttheofficialmaintainer(seetheMAINTAINERS
file).Alwaysgiveasmanyusefuldetailsaspossible.
329
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Howtosubmitpatchesordrivers
Don'tmergepatchesaddressingdifferentissues
Youshouldidentifyandcontacttheofficialmaintainer
forthefilestopatch.
SeeDocumentation/SubmittingPatchesfordetails.Fortrivial
patches,youcancopytheTrivialPatchMonkey.
Seealsohttp://kernelnewbies.org/UpstreamMergeforveryhelpful
advicetohaveyourcodemergedupstream(byRikvanRiel).
Patchesrelatedtoembeddedsystemscanbesubmittedonthelinux
embeddedmailinglist:
http://vger.kernel.org/vgerlists.html#linuxembedded
ARMplatform:it'sbesttosubmityourARMpatchestoRussellKing's
patchsystem:http://www.arm.linux.org.uk/developer/patches/
330
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Howtobecomeakerneldeveloper?
GregKroahHartmanwroteusefulguidelinesinthekernel
documentation:
Documentation/HOWTO
HowtoParticipateintheLinuxCommunity(byJonathanCorbet)
AGuideToTheKernelDevelopmentProcess
http://ldn.linuxfoundation.org/documentation/howparticipatelinuxcommunity
Readthislastdocumentfirst.
Itisprobablysufficient!
331
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Adviceandresources
References
332
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Specifictrainingmaterials
Moretrainingmaterialsavailableonhttp://freeelectrons.com/docs:
Newfeaturesin2.6
Kernelinitialization
PortingLinuxtonewhardware
Powermanagement
USBdevicedrivers
Blockdrivers
Morewillbeavailableinthenextmonths.
Don'thesitatetoaskustocreatetheonesyouneedforatraining
session!
333
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Informationsites(1)
LinuxWeeklyNews
http://lwn.net/
TheweeklydigestoffallLinuxandfreesoftwareinformation
sources
Indepthtechnicaldiscussionsaboutthekernel
Subscribetofinancetheeditors($5/month)
Articlesavailablefornonsubscribers
after1week.
334
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Informationsites(2)
KernelTrap
http://kerneltrap.org/
Forumwebsiteforkerneldevelopers
News,articles,whitepapers,discussions,polls,interviews
Perfectifadigestisnotenough!
335
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Usefulreading(1)
LinuxDeviceDrivers,3rdedition,Feb2005
ByJonathanCorbet,AlessandroRubini,
GregKroahHartman,O'Reilly
http://www.oreilly.com/catalog/linuxdrive3/
Freelyavailableonline!
Greatcompaniontotheprintedbook
foreasyelectronicsearches!
http://lwn.net/Kernel/LDD3/(1PDFfileperchapter)
http://freeelectrons.com/community/kernel/ldd3/(singlePDFfile)
AmusthavebookforLinuxdevicedriverwriters!
336
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Usefulreading(2)
EssentialLinuxDeviceDrivers,April2008
http://freeelectrons.com/redirect/elddbook.html
BySreekrishnanVenkateswaran,anembeddedIBM
engineerwithmorethan10yearsofexperience
CoversawiderangeoftopicsnotcoveredbyLDD:
serialdrivers,inputdrivers,I2C,PCMCIAand
CompactFlash,PCI,USB,videodrivers,audio
drivers,blockdrivers,networkdrivers,Bluetooth,
IrDA,MTD,driversinuserspace,kerneldebugging,
etc.
Probablythemostwiderangingandcomplete
LinuxdevicedriverbookI'vereadAlanCox
337
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Usefulreading(3)
LinuxKernelinaNutshell,Dec2006
ByGregKroahHartman,O'Reilly
http://www.kroah.com/lkn/
Agoodreferencebookandguideonconfiguring,
compilingandmanagingtheLinuxkernelsources.
Freelyavailableonline!
Greatcompaniontotheprintedbook
foreasyelectronicsearches!
AvailableassinglePDFfileon
http://freeelectrons.com/community/kernel/lkn/
338
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Usefulreading(4)
LinuxKernelDevelopment,2ndEdition,Jan2005
RobertLove,NovellPress
http://freeelectrons.com/redirect/lkd2book.html
Averysyntheticandpleasantwaytolearnaboutkernel
subsystems(beyondtheneedsofdevicedriverwriters)
UnderstandingtheLinuxKernel,3rdedition,Nov2005
DanielP.Bovet,MarcoCesati,O'Reilly
http://oreilly.com/catalog/understandlk/
AnextensivereviewofLinuxkernelinternals,
coveringLinux2.6atlast.
Unfortunately,onlycoversthePCarchitecture.
339
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Usefulonlineresources
LinuxkernelmailinglistFAQ
http://www.tux.org/lkml/
CompleteLinuxkernelFAQ
Readthisbeforeaskingaquestiontothemailinglist
KernelNewbies
http://kernelnewbies.org/
Glossary,articles,presentations,HOWTOs,
recommendedreading,usefultoolsforpeople
gettingfamiliarwithLinuxkernelordriverdevelopment.
Kernelglossary:
http://kernelnewbies.org/KernelGlossary
340
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxWiki
TheembeddedLinuxWikicontainsloadsofusefulresources
forembeddedsystemsdevelopers:
ManyHOWTOdocumentsofallkinds,coveringtopicslike
systemsize,boottime,multimedia,powermanagement,
toolchains...
Kernelpatchesnotavailableinmainstreamyet(e.g.LinuxTiny)
Communityresource:hackerinterviews,bookreviews,
eventcoverage...
Isopentoeveryone.Contributionsarewelcome!
http://elinux.org
341
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
ARMresources
ARMLinuxproject:http://www.arm.linux.org.uk/
Developerdocumentation:
http://www.arm.linux.org.uk/developer/
armlinuxkernelmailinglist:
http://lists.arm.linux.org.uk/mailman/listinfo/linuxarmkernel
FAQ:http://www.arm.linux.org.uk/armlinux/mlfaq.php
Howtopostkernelfixes:
http://www.arm.uk.linux.org/developer/patches/
ARMLinux@Simtec:http://armlinux.simtec.co.uk/
Afewusefulresources:FAQ,documentationandWho'swho!
ARMLimited:http://www.linuxarm.com/
Wikiwithlinkstousefuldeveloperresources
342
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Internationalconferences(1)
UsefulconferencesfeaturingLinuxkernelpresentations
OttawaLinuxSymposium(July):http://linuxsymposium.org/
Lotsofkerneltopicsbymajorkernelhackers.
Freelyavailableproceedings.
Fosdem:http://fosdem.org(Brussels,February)
Fordevelopers.Kernelpresentationsfromwellknownkernel
hackers.
EmbeddedLinuxConference:
http://embeddedlinuxconference.com/
OrganizedbytheCELinuxForum:California
(SanJose,April),inEurope(November).Frequenttechnical
sessionsinJapan.Veryinterestingkerneltopicsforembedded
systemsdevelopers.Presentationslidesfreelyavailable.
343
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Internationalconferences(2)
linux.conf.au:http://linux.org.au/conf/(Australia/NewZealand)
Featuresafewpresentationsbykeykernelhackers.
Don'tmissourfreeconferencevideoson
http://freeelectrons.com/community/videos/conferences/!
344
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Adviceandresources
Lastadvice
345
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
UsetheSource,Luke!
ManyresourcesandtricksontheInternetfindyouwill,butsolutions
toalltechnicalissuesonlyintheSourcelie.
ThankstoLucasArts
346
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Annexes
Quizanswers
347
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Quizanswers
Interrupthandling
Q:Whydidthekernelsegfaultatmoduleunload(forgettingto
unregisterahandlerinasharedinterruptline)?
A:Kernelmemoryisallocatedatmoduleloadtime,tohostmodule
code.Thismemoryisfreedatmoduleunloadtime.Ifyouforgetto
unregisterahandlerandaninterruptcomes,thecpuwilltrytojump
totheaddressofthehandler,whichisinafreedmemoryarea.
Crash!
348
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Annexes
Kernelsources
349
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Checkingtheintegrityofsources
KernelsourceintegritycanbecheckedthroughOpenPGPdigitalsignatures.
Fulldetailsonhttp://www.kernel.org/signature.html
DetailsaboutGnuPG:http://www.gnupg.org/gph/en/manual.html
ImportthepublicGnuPGkeyofkerneldevelopers:
gpgkeyserverpgp.mit.edurecvkeys0x517D0F0E
Ifblockedbyyourfirewall,lookfor0x517D0F0Eonhttp://pgp.mit.edu/,
copyandpastethekeytoalinuxkey.txtfile:
gpgimportlinuxkey.txt
Downloadthesignaturefilecorrespondingtoyourkernelversion
(atthesamelocation),andrunthesignaturecheck:
gpgverifylinux2.6.11.12.tar.bz2.sign
350
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Annexes
Slabcachesandmemorypools
351
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Slabcaches
Alsocalledlookasidecaches
Slabcaches:Objectsthatcanholdanynumber
ofmemoryareasofthesamesize.
OptimumuseofavailableRAM
andreducedfragmentation.
MainlyusedinLinuxcoresubsystems:filesystems(openfiles,
inodeandfilecaches...),networking...Livestatson
/proc/slabinfo.
Maybeusefulindevicedriverstoo,thoughnotusedsooften.
Linux2.6:usedbyUSBandSCSIdrivers.
352
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
SlabcacheAPI(1)
#include<linux/slab.h>
Creatingacache:
cache=kmem_cache_create(
name,
/*Namefor/proc/slabinfo*/
size,
/*Cacheobjectsize*/
align,
/*Cachealignment*/
flags,
/*Options:DMA,debugging,tracing...*/
constructor); /*Optional,calledaftereachallocation*/
Example:drivers/usb/host/uhcihcd.c
uhci_up_cachep=kmem_cache_create(
"uhci_urb_priv",sizeof(structurb_priv),
0,0,NULL);
353
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
SlabcacheAPI(2)
SinceLinux2.6.22,amacrocansimplifycachecreationinmostcases:
#defineKMEM_CACHE(__struct,__flags)\
kmem_cache_create(#__struct,\
sizeof(struct__struct),\
__alignof__(struct__struct),\
(__flags),NULL)
Example:kernel/pid.c
pid_cachep=KMEM_CACHE(pid,SLAB_PANIC);
354
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
SlabcacheAPI(3)
Allocatingfromthecache:
object=kmem_cache_alloc(cache,flags);
orobject=kmem_cache_zalloc(cache,flags);
Freeinganobject:
kmem_cache_free(cache,object);
Destroyingthewholecache:
kmem_cache_destroy(cache);
MoredetailsandanexampleintheLinuxDeviceDriversbook:
http://lwn.net/images/pdf/LDD3/ch08.pdf
355
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Memorypools
Usefulformemoryallocationsthatcannotfail
Kindoflookasidecachetryingtokeepaminimumnumberofpre
allocatedobjectsaheadoftime.
Usewithcare:otherwisecanresultinalotofunusedmemory
thatcannotbereclaimed!Useothersolutionswheneverpossible.
356
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
MemorypoolAPI(1)
#include<linux/mempool.h>
Mempoolcreation:
mempool=mempool_create(
min_nr,
alloc_function,
free_function,
pool_data);
357
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
MemorypoolAPI(2)
Allocatingobjects:
object=mempool_alloc(pool,flags);
Freeingobjects:
mempool_free(object,pool);
Resizingthepool:
status=mempool_resize(
pool,new_min_nr,flags);
Destroyingthepool(caution:freeallobjectsfirst!):
mempool_destroy(pool);
358
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Memorypoolimplementation
mempool_create
Callalloc
functionmin_nr
times
Callalloc
function
mempool_alloc
No
Success?
Takean
objectfrom
thepool
Yes
mempool_free
poolcount
<min_nr?
Yes
Addfreed
objecttopool
Newobject
No
Callfree
function
onobject
359
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Memorypoolsusingslabcaches
Idea:useslabcachefunctionstoallocateandfreeobjects.
Themempool_alloc_slabandmempool_free_slabfunctions
supplyalinkwithslabcacheroutines.
So,youwillfindmanycodeexampleslookinglike:
cache=kmem_cache_create(...);
pool=mempool_create(
min_nr,
mempool_alloc_slab,
mempool_free_slab,
cache);
There'sashorthandpoolcreationfunctionforthiscase:
pool=mempool_create_slab_pool(min_nr,cache);
360
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxdriverdevelopment
Annexes
Initrunlevels
361
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
SystemVinitrunlevels(1)
IntroducedbySystemVUnix
Muchmoreflexiblethanin
BSD
Makeitpossibletostartor
stopdifferentservicesforeach
runlevel
Correspondtotheargument
givento/sbin/init.
Runlevelsdefinedin
/etc/inittab.
/etc/initabexcerpt:
id:5:initdefault:
#Systeminitialization.
si::sysinit:/etc/rc.d/rc.sysinit
l0:0:wait:/etc/rc.d/rc0
l1:1:wait:/etc/rc.d/rc1
l2:2:wait:/etc/rc.d/rc2
l3:3:wait:/etc/rc.d/rc3
l4:4:wait:/etc/rc.d/rc4
l5:5:wait:/etc/rc.d/rc5
l6:6:wait:/etc/rc.d/rc6
362
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
SystemVinitrunlevels(2)
Standardlevels
init0
Haltthesystem
init1
Singleusermodeformaintenance
init6
Rebootthesystem
Customizablelevels:2,3,4,5
init3
Oftenmultiusermode,with
onlycommandlinelogin
init5
Oftenmultiusermode,with
graphicallogin
initS
Singleusermodefor
maintenance.
Mountingonly/.
Oftenidenticalto1
363
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
initscripts
Accordingto/etc/inittabsettings,init<n>runs:
First/etc/rc.d/rc.sysinitforallrunlevels
Thenscriptsin/etc/rc<n>.d/
Startingservices(1,3,5,S):
runsS*scriptswiththestartoption
Killingservices(0,6):
runsK*scriptswiththestopoption
Scriptsareruninfilenamelexicalorder
Justuselsltofindouttheorder!
364
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
/etc/init.d
Repositoryforallavailableinitscripts
/etc/rc<n>.d/onlycontainslinkstothe/etc/init.d/
scriptsneededforrunleveln
/etc/rc1.d/example(fromFedoraCore3)
K01yum>../init.d/yum
K02cupsconfigdaemon>../init.d/cups
configdaemon
K02haldaemon>../init.d/haldaemon
K02NetworkManager>
../init.d/NetworkManager
K03messagebus>../init.d/messagebus
K03rhnsd>../init.d/rhnsd
K05anacron>../init.d/anacron
K05atd>../init.d/atd
S00single>../init.d/single
S01sysstat>../init.d/sysstat
S06cpuspeed>../init.d/cpuspeed
365
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Handlinginitscriptsbyhand
Simplycallthe/etc/init.dscripts!
sudo/etc/init.d/sshdstart
Startingsshd:[OK]
sudo/etc/init.d/nfsstop
ShuttingdownNFSmountd:[FAILED]
ShuttingdownNFSdaemon:
[FAILED]ShuttingdownNFSquotas:
[FAILED]
ShuttingdownNFSservices:[OK]
sudo/etc/init.d/pcmciastatus
cardmgr(pid3721)isrunning...
sudo/etc/init.d/httpdrestart
Stoppinghttpd:[OK]
Startinghttpd:[OK]
366
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
InitrunlevelsUsefullinks
Backtotheslideabouttheinitprogram.
367
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Traininglabs
Traininglabsarealsoavailablefromthesamelocation:
http://freeelectrons.com/training/drivers
Theyareausefulcomplementtoconsolidatewhatyou
learnedfromthistraining.Theydon'ttellhowtodothe
exercises.However,theyonlyrelyonnotionsandtools
introducedbythelectures.
Ifyouhappentobestuckwithanexercise,thisprovesthat
youmissedsomethinginthelecturesandhavetogobackto
theslidestofindwhatyou'relookingfor.
368
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Relateddocuments
AllthetechnicalpresentationsandtrainingmaterialscreatedandusedbyFree
Electrons,availableunderafreedocumentationlicense(morethan1500pages!).
http://freeelectrons.com/training
LinuxUSBdrivers
RealtimeinembeddedLinuxsystems
IntroductiontoUnixandGNU/Linux
EmbeddedLinuxkernelanddriverdevelopment IntroductiontouClinux
FreeSoftwaretoolsforembeddedLinuxsystems LinuxonTIOMAPprocessors
FreeSoftwaredevelopmenttools
AudioinembeddedLinuxsystems
JavainembeddedLinuxsystems
MultimediainembeddedLinuxsystems
IntroductiontoGNU/LinuxandFreeSoftwar
http://freeelectrons.com/articles
Linuxandecology
What'snewinLinux2.6?
AdvantagesofFreeSoftwareinembeddedsystems
HowtoportLinuxonanewPDA
EmbeddedLinuxoptimizations
EmbeddedLinuxfromScratch...in40min!
369
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Howtohelp
Ifyousupportthiswork,youcanhelp...
Bysendingcorrections,suggestions,contributionsand
translations
Byaskingyourorganizationtoordertrainingsessions
performedbytheauthorofthesedocuments(see
http://freeelectrons.com/training)
Byspeakingaboutittoyourfriends,colleagues
andlocalFreeSoftwarecommunity.
Byaddinglinkstoouronlinematerialsonyourwebsite,
toincreasetheirvisibilityinsearchengineresults.
370
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
Thanks
TotheOpenOffice.orgproject,fortheirpresentation
andwordprocessortoolswhichsatisfiedallmyneeds
Tohttp://openclipart.orgprojectcontributorsfortheir
nicepublicdomainclipart
TothemembersofthewholeFreeSoftwareandOpen
Sourcecommunity,forsharingthebestofthemselves:
theirwork,theirknowledge,theirfriendship.
ToBillGates,forleavinguswithsomuchroomfor
innovation!
Topeoplewho
helped,sent
correctionsor
suggestions:
FlorentPeyraud
MarcZyngier
VanessaConchodon,
StphaneRubino,
SamuliJarvinen,Phil
Blundell,JefferyHuang,
MohitMehta,Matti
Aaltonen,RobertP.J.
Day
371
FreeElectrons.Kernel,driversandembeddedLinuxdevelopment,consulting,trainingandsupport.http//freeelectrons.com
EmbeddedLinuxTraining
UnixandGNU/Linuxbasics
Linuxkernelanddriversdevelopment
RealtimeLinux
uClinux
Developmentandprofilingtools
Lightweighttoolsforembeddedsystems
Rootfilesystemcreation
Audioandmultimedia
Systemoptimization
Consulting
Helpindecisionmaking
Systemarchitecture
Identificationofsuitabletechnologies
Managinglicensingrequirements
Systemdesignandperformancereview
http://freeelectrons.com
FreeElectronsservices
CustomDevelopment
Systemintegration
EmbeddedLinuxdemosandprototypes
Systemoptimization
Linuxkerneldrivers
Applicationandinterfacedevelopment
TechnicalSupport
Developmenttoolandapplicationsupport
Issueinvestigationandsolutionfollowupwith
mainstreamdevelopers
Helpgettingstarted