William's Blog with Octopress

Octopress is A blogging framework for hackers.

Chef Ohai源码学习

| Comments

接触Chef以来一直对Ohai很感兴趣,它收集的系统信息非常全面,很好奇它是怎么收集的,现在终于有时间学习了

目标

搞清楚Ohai是如何收集到某一项系统消息的,如IP地址

环境搭建

使用gem install ohai安装ohai和它的依赖,然后下载ohai的源码

git clone https://github.com/opscode/ohai.git

可以直接在源码目录通过./bin/ohai运行ohai,这样可以对源码做一些修改(添加调试代码)然后马上运行查看效果,而不用去找它的源码安装到系统的位置

还可以使用pry进行单步调试: gem install pry安装pry,的要调试的代码前面加上binding.pry,在该文件前面加上require 'pry'

查看源码过程

首先找到入口,当然就是bin/ohai文件了,我们需要关系的只有一行

bin/ohai
1
Ohai::Application.new.run

我们在看一下Ohai::Application的定义,我们感兴趣的是run方法的定义

lib/ohai/application.rb
1
2
3
4
5
def run
  configure_ohai
  configure_logging
  run_application
end

这里前两个方法我们目前不关心,看一下run_application方法的定义

lib/ohai/application.rb
1
2
3
4
5
6
7
8
9
10
11
12
def run_application
  ohai = Ohai::System.new
  ohai.all_plugins(@attributes)

  if @attributes
    @attributes.each do |a|
      puts ohai.attributes_print(a)
    end
  else
    puts ohai.json_pretty_print
  end
end

通过加调试代码发现这里的@attributes是nil,所以这里的代码可以简化为

1
2
3
ohai = Ohai::System.new
ohai.all_plugins(nil)
puts ohai.json_pretty_print

下来去看Ohai::System的定义

首先我们看一下json_pretty_print的定义

lib/ohai/system.rb
1
2
3
def json_pretty_print(item=nil)
  Yajl::Encoder.new(:pretty => true).encode(item || @data)
end

它只是把@data的数据格式化后输出,所以我们推测实际收集的动作是发生在all_plugins方法里

lib/ohai/system.rb
1
2
3
4
5
6
7
8
9
def all_plugins(attribute_filter=nil)
  # Reset the system when all_plugins is called since this function
  # can be run multiple times in order to pick up any changes in the
  # config or plugins with Chef.
  reset_system

  load_plugins
  run_plugins(true, attribute_filter)
end

reset_system方法里只是初始化了一些实例变量,其中包括@loader@runner

1
2
@loader = Ohai::Loader.new(self)
@runner = Ohai::Runner.new(self, true)

这里把self传了进去,这里的self就就Ohai::System的实例,我们看一下loader如何处理

lib/ohai/loader.rb
1
2
3
4
5
def initialize(controller)
  @controller = controller
  @v6_plugin_classes = []
  @v7_plugin_classes = []
end

loader把Ohai::System的实例保存到了实例变量@controller

load_plugins方法只有一行,调用了@loaderload_all方法,我们看一下这个方法

lib/ohai/loader.rb
1
2
3
4
5
6
7
8
def load_all
  plugin_files_by_dir.each do |plugin_file|
    load_plugin_class(plugin_file.path, plugin_file.plugin_root)
  end

  collect_v6_plugins
  collect_v7_plugins
end

看一下plugin_files_by_dir的定义

lib/ohai/loader.rb
1
2
3
4
5
def plugin_files_by_dir
  Array(Ohai::Config[:plugin_path]).inject([]) do |plugin_files, plugin_path|
    plugin_files + PluginFile.find_all_in(plugin_path)
  end
end

注释中说它搜索所有的plugin路径并返回一个包含PluginFile对象的数组

在看一下PluginFile对象长什么样子

lib/ohai/loader.rb
1
2
3
4
5
6
7
8
class PluginFile < Struct.new(:path, :plugin_root)

  def self.find_all_in(plugin_dir)
    Dir[File.join(plugin_dir, "**", "*.rb")].map do |file|
      new(file, plugin_dir)
    end
  end
end

可以看到它只是简单的从Struct继承而来,这里所起的作用就是定义了两个访问器pathplugin_root,打印出来类似这样

lib/ohai/loader.rb
1
2
3
#<struct Ohai::Loader::PluginFile
 path="/Users/william/Codes/ohai/lib/ohai/plugins/aix/cpu.rb",
 plugin_root="/Users/william/Codes/ohai/lib/ohai/plugins">

再回到load_all方法

lib/ohai/loader.rb
1
2
3
4
5
6
7
8
def load_all
  plugin_files_by_dir.each do |plugin_file|
    load_plugin_class(plugin_file.path, plugin_file.plugin_root)
  end

  collect_v6_plugins
  collect_v7_plugins
end

对每一个PluginFile对象调用了load_plugin_class方法

lib/ohai/loader.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
def load_plugin_class(plugin_path, plugin_dir_path=nil)
  # Read the contents of the plugin to understand if it's a V6 or V7 plugin.
  contents = ""
  begin
    contents << IO.read(plugin_path)
  rescue IOError, Errno::ENOENT
    Ohai::Log.warn("Unable to open or read plugin at #{plugin_path}")
    return nil
  end

  # We assume that a plugin is a V7 plugin if it contains Ohai.plugin in its contents.
  if contents.include?("Ohai.plugin")
    load_v7_plugin_class(contents, plugin_path)
  else
    Ohai::Log.warn("[DEPRECATION] Plugin at #{plugin_path} is a version 6 plugin. \
Version 6 plugins will not be supported in future releases of Ohai. \
Please upgrade your plugin to version 7 plugin syntax. \
For more information visit here: docs.opscode.com/ohai_custom.html")

    load_v6_plugin_class(contents, plugin_path, plugin_dir_path)
  end
end

他把文件里的内容读了进来,并根据有没有包含Ohai.plugin来有选择了调用load_v7_plugin_classload_v6_plugin_class,我大概看来一下基本上全都包含Ohai.plugin,所以我们从v7追

lib/ohai/loader.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
def load_v7_plugin_class(contents, plugin_path)
  plugin_class = eval(contents, TOPLEVEL_BINDING, plugin_path)
  unless plugin_class.kind_of?(Class) and plugin_class < Ohai::DSL::Plugin
    raise Ohai::Exceptions::IllegalPluginDefinition, "Plugin file cannot contain any statements after the plugin definition"
  end
  plugin_class.sources << plugin_path
  @v7_plugin_classes << plugin_class unless @v7_plugin_classes.include?(plugin_class)
  plugin_class
rescue SystemExit, Interrupt
  raise
rescue Ohai::Exceptions::InvalidPluginName => e
  Ohai::Log.warn("Plugin Name Error: <#{plugin_path}>: #{e.message}")
rescue Ohai::Exceptions::IllegalPluginDefinition => e
  Ohai::Log.warn("Plugin Definition Error: <#{plugin_path}>: #{e.message}")
rescue NoMethodError => e
  Ohai::Log.warn("Plugin Method Error: <#{plugin_path}>: unsupported operation \'#{e.name}\'")
rescue SyntaxError => e
  # split on occurrences of
  #    <env>: syntax error,
  #    <env>:##: syntax error,
  # to remove from error message
  parts = e.message.split(/<.*>[:[0-9]+]*: syntax error, /)
  parts.each do |part|
    next if part.length == 0
    Ohai::Log.warn("Plugin Syntax Error: <#{plugin_path}>: #{part}")
  end
rescue Exception, Errno::ENOENT => e
  Ohai::Log.warn("Plugin Error: <#{plugin_path}>: #{e.message}")
  Ohai::Log.debug("Plugin Error: <#{plugin_path}>: #{e.inspect}, #{e.backtrace.join('\n')}")
end

异常处理我们不关心,直接删掉来看

lib/ohai/loader.rb
1
2
3
4
5
6
7
8
9
def load_v7_plugin_class(contents, plugin_path)
  plugin_class = eval(contents, TOPLEVEL_BINDING, plugin_path)
  unless plugin_class.kind_of?(Class) and plugin_class < Ohai::DSL::Plugin
    raise Ohai::Exceptions::IllegalPluginDefinition, "Plugin file cannot contain any statements after the plugin definition"
  end
  plugin_class.sources << plugin_path
  @v7_plugin_classes << plugin_class unless @v7_plugin_classes.include?(plugin_class)
  plugin_class
end

这里第一行把插件文件里的代码执行,返回的结果赋给plugin_class,把plugin_path保存到其中,并把它收集到实例变量@v7_plugin_classes

我们找最简单了plugin看里面是什么

lib/ohai/plugins/command.rb
1
2
3
4
5
6
7
Ohai.plugin(:Command) do
  provides "command"

  collect_data do
    command Mash.new
  end
end

它是调用Ohaiplugin方法,还传给它一个block,看看这个方法的定义

lib/ohai/dsl/plugin.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
def self.plugin(name, &block)
  raise Ohai::Exceptions::InvalidPluginName, "#{name} is not a valid plugin name. A valid plugin name is a symbol which begins with a capital letter and contains no underscores" unless NamedPlugin.valid_name?(name)

  plugin = nil

  if NamedPlugin.strict_const_defined?(name)
    plugin = NamedPlugin.const_get(name)
    plugin.class_eval(&block)
  else
    klass = Class.new(DSL::Plugin::VersionVII, &block)
    plugin = NamedPlugin.const_set(name, klass)
  end

  plugin
end

这个方法重点是这两行

lib/ohai/dsl/plugin.rb
1
2
klass = Class.new(DSL::Plugin::VersionVII, &block)
plugin = NamedPlugin.const_set(name, klass)

第一行用传进来了block创建了一个继承自DSL::Plugin::VersionVII的类,然后一传进来的name为常量名保存到模块NamedPlugin

我们在回到前面的plugin文件

lib/ohai/plugins/command.rb
1
2
3
4
5
6
7
Ohai.plugin(:Command) do
  provides "command"

  collect_data do
    command Mash.new
  end
end

这里还有两个方法providescollect_data,你没有猜错,它们就是定义在继承来的DSL::Plugin::VersionVII

lib/ohai/dsl/plugin/versionvii.rb
1
2
3
4
5
def self.provides(*attrs)
  attrs.each do |attr|
    provides_attrs << attr unless provides_attrs.include?(attr)
  end
end

它只是它传进来的参数收集到provides_atts,而provides_atts是前面生成的类的实例变量

lib/ohai/dsl/plugin/versionvii.rb
1
2
3
def self.provides_attrs
  @provides_attrs ||= []
end

在看collect_data

lib/ohai/dsl/plugin/versionvii.rb
1
2
3
4
5
6
7
8
9
def self.collect_data(platform = :default, *other_platforms, &block)
  [platform, other_platforms].flatten.each do |plat|
    if data_collector.has_key?(plat)
      raise Ohai::Exceptions::IllegalPluginDefinition, "collect_data already defined on platform #{plat}"
    else
      data_collector[plat] = block
    end
  end
end

由于前面的plugin中调用这个方法的时候只传了一个快进来,所从这个方法只是把传进来了块赋值给了以:default为key的Mash对象data_collector

lib/ohai/dsl/plugin/versionvii.rb
1
2
3
def self.data_collector
  @data_collector ||= Mash.new
end

我们再一次回到load_all方法中

lib/ohai/loader.rb
1
2
3
4
5
6
7
8
def load_all
  plugin_files_by_dir.each do |plugin_file|
    load_plugin_class(plugin_file.path, plugin_file.plugin_root)
  end

  collect_v6_plugins
  collect_v7_plugins
end

还剩下两行代码,我们只看一下collect_v7_plugins

lib/ohai/loader.rb
1
2
3
4
5
def collect_v7_plugins
  @v7_plugin_classes.each do |plugin_class|
    load_v7_plugin(plugin_class)
  end
end

对收集的plugin_class调用load_v7_plugin方法

lib/ohai/loader.rb
1
2
3
4
5
def load_v7_plugin(plugin_class)
  plugin = plugin_class.new(@controller.data)
  collect_provides(plugin)
  plugin
end

这个方法把传进来的类实例化,并把Ohai::System实例的data传了进去,然后调用了collect_provides

lib/ohai/loader.rb
1
2
3
4
def collect_provides(plugin)
  plugin_provides = plugin.class.provides_attrs
  @controller.provides_map.set_providers_for(plugin, plugin_provides)
end

还记得前面最简单的plugin的代码吗,这里的plugin.class.provides_attrs就是provides后面的参数(“command”)

这里的provides_map是在Ohai::Systemreset_system赋值的,是ProvidesMap的实例

lib/ohai/loader.rb
1
@provides_map = ProvidesMap.new

然后它又调用了set_providers_for方法

lib/ohai/loader.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
def set_providers_for(plugin, provided_attributes)
  unless plugin.kind_of?(Ohai::DSL::Plugin)
    raise ArgumentError, "set_providers_for only accepts Ohai Plugin classes (got: #{plugin})"
  end

  provided_attributes.each do |attribute|
    attrs = @map
    parts = normalize_and_validate(attribute)
    parts.each do |part|
      attrs[part] ||= Mash.new
      attrs = attrs[part]
    end
    attrs[:_plugins] ||= []
    attrs[:_plugins] << plugin
  end
end

这里把传来的keys它plugin收集到了providesmap的实例变量@map中,类似这样

lib/ohai/loader.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
[2] pry(#<Ohai::ProvidesMap>)> @map
=> {"cpu"=>
     {"_plugins"=>
       [#<Ohai::NamedPlugin::CPU:0x00000101a6a4d0
         @data={},
         @has_run=false,
         @source=
           ["/Users/william/Codes/ohai/lib/ohai/plugins/aix/cpu.rb",
             "/Users/william/Codes/ohai/lib/ohai/plugins/darwin/cpu.rb",
             "/Users/william/Codes/ohai/lib/ohai/plugins/freebsd/cpu.rb",
             "/Users/william/Codes/ohai/lib/ohai/plugins/linux/cpu.rb",
             "/Users/william/Codes/ohai/lib/ohai/plugins/netbsd/cpu.rb",
             "/Users/william/Codes/ohai/lib/ohai/plugins/openbsd/cpu.rb",
             "/Users/william/Codes/ohai/lib/ohai/plugins/sigar/cpu.rb",
             "/Users/william/Codes/ohai/lib/ohai/plugins/solaris2/cpu.rb",
             "/Users/william/Codes/ohai/lib/ohai/plugins/windows/cpu.rb"
           ],
         @version=:version7>
       ]
     }
   }

我们再回到all_plugins方法,前面是load_plugins方法的深入执行过程,现在来看run_plugins方法

lib/ohai/system.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
def run_plugins(safe = false, attribute_filter = nil)
  # First run all the version 6 plugins
  @v6_dependency_solver.values.each do |v6plugin|
    @runner.run_plugin(v6plugin)
  end

  # Then run all the version 7 plugins
  begin
    @provides_map.all_plugins(attribute_filter).each { |plugin|
      @runner.run_plugin(plugin)
    }
  rescue Ohai::Exceptions::AttributeNotFound, Ohai::Exceptions::DependencyCycle => e
    Ohai::Log.error("Encountered error while running plugins: #{e.inspect}")
    raise
  end
end

我们只看v7的,这里比较简单,对每一个plugin调用@runnerrun_plugin方法(@runner)是在reset_system中定义的

lib/ohai/system.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
def run_plugin(plugin)
  unless plugin.kind_of?(Ohai::DSL::Plugin)
    raise Ohai::Exceptions::InvalidPlugin, "Invalid plugin #{plugin} (must be an Ohai::DSL::Plugin or subclass)"
  end

  if Ohai::Config[:disabled_plugins].include?(plugin.name)
    Ohai::Log.debug("Skipping disabled plugin #{plugin.name}")
    return false
  end

  begin
    case plugin.version
    when :version7
      run_v7_plugin(plugin)
    when :version6
      run_v6_plugin(plugin)
    else
      raise Ohai::Exceptions::InvalidPlugin, "Invalid plugin version #{plugin.version} for plugin #{plugin}"
    end
  rescue Ohai::Exceptions::Error
    raise
  rescue Exception,Errno::ENOENT => e
    Ohai::Log.debug("Plugin #{plugin.name} threw exception #{e.inspect} #{e.backtrace.join("\n")}")
  end
end

简单说它就是转而去调用run_v7_plugin

lib/ohai/system.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
def run_v7_plugin(plugin)
  visited = [ plugin ]
  while !visited.empty?
    next_plugin = visited.pop

    next if next_plugin.has_run?

    if visited.include?(next_plugin)
      raise Ohai::Exceptions::DependencyCycle, "Dependency cycle detected. Please refer to the following plugins: #{get_cycle(visited, plugin).join(", ") }"
    end

    dependency_providers = fetch_plugins(next_plugin.dependencies)

    # Remove the already ran plugins from dependencies if force is not set
    # Also remove the plugin that we are about to run from dependencies as well.
    dependency_providers.delete_if { |dep_plugin|
      dep_plugin.has_run? || dep_plugin.eql?(next_plugin)
    }

    if dependency_providers.empty?
      @safe_run ? next_plugin.safe_run : next_plugin.run
    else
      visited << next_plugin << dependency_providers.first
    end
  end
end

简单说就是去调用plugin自己的safe_run方法(因为在定义@runner的时候有传第二个参数true)

lib/ohai/system.rb
1
2
3
4
5
6
7
8
9
10
def safe_run
  begin
    self.run
  rescue Ohai::Exceptions::Error => e
    raise e
  rescue => e
    Ohai::Log.debug("Plugin #{self.name} threw #{e.inspect}")
    e.backtrace.each { |line| Ohai::Log.debug( line )}
  end
end

它又调用了run方法

lib/ohai/system.rb
1
2
3
4
def run
  @has_run = true
  run_plugin
end

它又调用了run_plugin方法,这个方法是在lib/ohai/dsl/plugin/versionvii.rb中定义的

lib/ohai/system.rb
1
2
3
4
5
6
7
8
9
10
11
12
def run_plugin
  collector = self.class.data_collector
  platform = collect_os

  if collector.has_key?(platform)
    self.instance_eval(&collector[platform])
  elsif collector.has_key?(:default)
    self.instance_eval(&collector[:default])
  else
    Ohai::Log.debug("No data to collect for plugin #{self.name}. Continuing...")
  end
end

它用instance_eval执行了前面动态构造Plugin类的时候保存下来的块(collect_data后面跟的块)

整个过程过了一遍,但有一点还没明白,最开始的地方我们发现显示出来的信息都收集在Ohai::System的实例@ohai的实例变量@data里,执行插件里的代码怎么会修改到它呢?

这是因为在实例化plugin的时候它把实例变量@data传了进去

lib/ohai/loader.rb
1
2
3
4
5
def load_v7_plugin(plugin_class)
  plugin = plugin_class.new(@controller.data)
  collect_provides(plugin)
  plugin
end

在看一下plugin的部分代码

lib/ohai/loader.rb
1
2
3
4
5
6
7
Ohai.plugin(:Command) do
  provides "command"

  collect_data do
    command Mash.new
  end
end

传给collect_data的块中的command是不是很奇怪,它是个什么东西?

答案的下面的代码中

lib/ohai/dsl/plugin.rb
1
2
3
4
5
6
7
8
9
def method_missing(name, *args)
  return get_attribute(name) if args.length == 0

  set_attribute(name, *args)
end

def get_attribute(name)
  @data[name]
end

到此所有迷雾都解开了, Yeah!!!

Comments