Skip to content
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 4 additions & 60 deletions lib/vmpooler/pool_manager.rb
Original file line number Diff line number Diff line change
Expand Up @@ -224,10 +224,9 @@ def _clone_vm(pool, provider)
$logger.log('s', "[+] [#{pool_name}] '#{new_vmname}' cloned in #{finish} seconds")

$metrics.timing("clone.#{pool_name}", finish)
rescue => err
$logger.log('s', "[!] [#{pool_name}] '#{new_vmname}' clone failed with an error: #{err}")
rescue => _err
$redis.srem('vmpooler__pending__' + pool_name, new_vmname)
raise
raise _err
ensure
$redis.decr('vmpooler__tasks__clone')
end
Expand Down Expand Up @@ -459,69 +458,17 @@ def _check_snapshot_queue
end
end

def migration_limit(migration_limit)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the migration limit is now a per vSphere provider setting the documentation should be updated too.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It probably should be a per vSphere provider setting, but it's still tracking migrations in redis and getting the setting from the global configuration section instead of checking the provider for this setting. I could add this change here. I was considering tackling that and the task_limit concept in a future change.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be as simple as add in the docs migration limit (Only affects vSphere Provider)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see what you're saying, not that it's a per provider setting, but rather that it is only a vsphere provider setting.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah... The docs suggest its global but no longer.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a mention in the example configuration file. I couldn't find another reference to migration_limit, please let me know if I've missed another location this should go.

# Returns migration_limit setting when enabled
return false if migration_limit == 0 || !migration_limit # rubocop:disable Style/NumericPredicate
migration_limit if migration_limit >= 1
end

def migrate_vm(vm_name, pool_name, provider)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is currently no way in Pool Manager to actually add a VM to the migration queue. So this PR will not have any affect yet. Is this by design?

@mattkirby mattkirby Nov 3, 2017

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's an oversight. It's fixed now.

Thread.new do
begin

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The provider really shouldn't need to contact redis for this. In this case the provider is just removing it from the queue. So the $redis.srem('vmpooler__migrating__' + pool_name, vm_name) should be called here PRIOR to the provider.migrate_vm call

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took your suggestion and placed this prior to calling provider.migrate_vm.

_migrate_vm(vm_name, pool_name, provider)
$redis.srem("vmpooler__migrating__#{pool_name}", vm_name)
provider.migrate_vm(pool_name, vm_name)
rescue => err
$logger.log('s', "[x] [#{pool_name}] '#{vm_name}' migration failed with an error: #{err}")
remove_vmpooler_migration_vm(pool_name, vm_name)
end
end
end

def _migrate_vm(vm_name, pool_name, provider)
$redis.srem('vmpooler__migrating__' + pool_name, vm_name)

parent_host_name = provider.get_vm_host(pool_name, vm_name)
raise('Unable to determine which host the VM is running on') if parent_host_name.nil?
migration_limit = migration_limit $config[:config]['migration_limit']
migration_count = $redis.scard('vmpooler__migration')

if !migration_limit
$logger.log('s', "[ ] [#{pool_name}] '#{vm_name}' is running on #{parent_host_name}")
return
elsif migration_count >= migration_limit
$logger.log('s', "[ ] [#{pool_name}] '#{vm_name}' is running on #{parent_host_name}. No migration will be evaluated since the migration_limit has been reached")
return
else
$redis.sadd('vmpooler__migration', vm_name)
host_name = provider.find_least_used_compatible_host(pool_name, vm_name)
if host_name == parent_host_name
$logger.log('s', "[ ] [#{pool_name}] No migration required for '#{vm_name}' running on #{parent_host_name}")
else
finish = migrate_vm_and_record_timing(vm_name, pool_name, parent_host_name, host_name, provider)
$logger.log('s', "[>] [#{pool_name}] '#{vm_name}' migrated from #{parent_host_name} to #{host_name} in #{finish} seconds")
end
remove_vmpooler_migration_vm(pool_name, vm_name)
end
end

def remove_vmpooler_migration_vm(pool, vm)
$redis.srem('vmpooler__migration', vm)
rescue => err
$logger.log('s', "[x] [#{pool}] '#{vm}' removal from vmpooler__migration failed with an error: #{err}")
end

def migrate_vm_and_record_timing(vm_name, pool_name, source_host_name, dest_host_name, provider)
start = Time.now
provider.migrate_vm_to_host(pool_name, vm_name, dest_host_name)
finish = format('%.2f', Time.now - start)
$metrics.timing("migrate.#{pool_name}", finish)
$metrics.increment("migrate_from.#{source_host_name}")
$metrics.increment("migrate_to.#{dest_host_name}")
checkout_to_migration = format('%.2f', Time.now - Time.parse($redis.hget("vmpooler__vm__#{vm_name}", 'checkout')))
$redis.hset("vmpooler__vm__#{vm_name}", 'migration_time', finish)
$redis.hset("vmpooler__vm__#{vm_name}", 'checkout_to_migration', checkout_to_migration)
finish
end

# Helper method mainly used for unit testing
def time_passed?(_event, time)
Time.now > time
Expand Down Expand Up @@ -764,9 +711,6 @@ def _check_pool(pool, provider)
end

pool_check_response
rescue => err
$logger.log('d', "[!] [#{pool['name']}] _check_pool failed with an error: #{err}")
raise
end

# Create a provider object, usually based on the providers/*.rb class, that implements providers/base.rb
Expand Down
8 changes: 8 additions & 0 deletions lib/vmpooler/providers/base.rb
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,14 @@ def migrate_vm_to_host(_pool_name, _vm_name, _dest_host_name)
raise("#{self.class.name} does not implement migrate_vm_to_host")
end

# inputs
# [String] pool_name : Name of the pool
# [String] vm_name : Name of the VM to migrate
# [Class] redis : Redis object
def migrate_vm(_pool_name, _vm_name, _redis)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart from Redis, how is this method signature different from migrate_vm_to_host?. Is it just simply calling

migrate_vm_to_host(pool_name, vm_name, nil) ?

Redis is already a global variable, so no ed to pass it through. That should probably be a separate PR

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main difference to me, beyond the redis part which as you point out shouldn't be there, is that there's no longer a need to provide the destination host name, so I figured keeping migrate_vm instead of migrate_vm_to_host made sense. I can change it if it makes more sense.

raise("#{self.class.name} does not implement migrate_vm")
end

# inputs
# [String] pool_name : Name of the pool
# [String] vm_name : Name of the VM to find
Expand Down
Loading