Distributed Ruby and Rails
Distributed Ruby
- DRb
- Rinda
- Starfish
- MapReduce
- MagLev VM
DRb
- Ruby's RMI (remote method invocation) system
- An object in one Ruby process can invoke methods on an object in another Ruby process on the same or a different machine.
- No defined interface, faster development time
- Tightly couple applications, because no defined API, but rather method on objects
- Unreliable under large-scale, heavy loads production environments
Server Example 1
require 'drb'
class HelloWorldServer
def say_hello
'Hello, world!'
end
end
DRb.start_service("druby://127.0.0.1:61676",
HelloWorldServer.new)
DRb.thread.joinClicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
Client Example 1
require 'drb'
server = DRbObject.new_with_uri("druby://127.0.0.1:61676")
puts server.say_hello
puts server.inspect
# Hello, world!
# <DRb::DRbObject:0x1003c04c8 @ref=nil, @uri="druby://
127.0.0.1:61676">Clicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
Example 2
# user.rb
class User
attr_accessor :username
endClicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
Server example 2
require 'drb'
require 'user'
class UserServer
attr_accessor :users
def find(id)
self.users[id-1]
end
end
user_server = UserServer.new
user_server.users = []
5.times do |i|
user = User.new
user.username = i + 1
user_server.users << user
end
DRb.start_service("druby://127.0.0.1:61676", user_server)
DRb.thread.joinClicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
Client Example 2
require 'drb'
user_server = DRbObject.new_with_uri("druby://127.0.0.1:61676")
user = user_server.find(2)
puts user.inspect
puts "Username: #{user.username}"
user.name = "ihower"
puts "Username: #{user.username}"Clicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
Err
# <DRb::DRbUnknown:0x1003b8318 @name="User", @buf="\004\bo:
\tUser\006:\016@usernamei\a">
# client2.rb:8: undefined method `username' for
#<DRb::DRbUnknown:0x1003b8318> (NoMethodError)Why DRbUndumped ?
- Default DRb operation
- Pass by value
- Must share code
- With DRbUndumped
- Pass by reference
- No need to share code
Example 2 Fixed
# user.rb
class User
include DRbUndumped
attr_accessor :username
end
# <DRb::DRbObject:0x1003b84f8 @ref=2149433940,
@uri="druby://127.0.0.1:61676">
# Username: 2
# Username: ihowerClicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
Why use DRbUndumped?
- Big objects
- Singleton objects
- Lightweight clients
- Rapidly changing software
ID conversion
- Converts reference into DRb object on server
- DRbIdConv (Default)
- TimerIdConv
- NamedIdConv
- GWIdConv
DRb security
require 'drb'
ro = DRbObject.new_with_uri("druby://127.0.0.1:61676")
class << ro
undef :instance_eval
end
# !!!!!!!! WARNING !!!!!!!!! DO NOT RUN
ro.instance_eval("`rm -rf *`")Clicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
$SAFE=1
instance_eval':
Insecure operation - instance_eval (SecurityError)Clicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
- Access Control Lists (ACLs)
- via IP address array
- still can run denial-of-service attack
- DRb over SSL
Rinda
- Rinda is a Ruby port of Linda distributed computing paradigm.
- Linda is a model of coordination and communication among several parallel processes operating upon objects stored in and retrieved from shared, virtual, associative memory. This model is implemented as a "coordination language" in which several primitives operating on ordered sequence of typed data objects, "tuples," are added to a sequential language, such as C, and a logically global associative memory, called a tuplespace, in which processes store and retrieve tuples
- Rinda consists of:
- A TupleSpace implementation
- A RingServer that allows DRb services to
- automatically discover each other.
RingServer
- We hardcoded IP addresses in DRb program, it’s tight coupling of applications and make fault tolerance difficult.
- RingServer can detect and interact with other services on the network without knowing IP addresses.
Learn Ruby on Rails - Ruby on Rails tutorial - Ring Server - Ruby on Rails examples - Ruby On Rails programs
Ring Server Example
require 'rinda/ring'
require 'rinda/tuplespace'
DRb.start_service
Rinda::RingServer.new(Rinda::TupleSpace.new)
DRb.thread.joinClicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
Service Example
require 'rinda/ring'
class HelloWorldServer
include DRbUndumped # Need for RingServer
def say_hello
'Hello, world!'
end
end
DRb.start_service
ring_server = Rinda::RingFinger.primary
ring_server.write([:hello_world_service, :HelloWorldServer, HelloWorldServer.new,
'I like to say hi!'], Rinda::SimpleRenewer.new)
DRb.thread.joinClicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
Client Example
require 'rinda/ring'
DRb.start_service
ring_server = Rinda::RingFinger.primary
service = ring_server.read([:hello_world_service, nil,nil,nil])
server = service[2]
puts server.say_hello
puts service.inspect
# Hello, world!
# [:hello_world_service, :HelloWorldServer, #<DRb::DRbObject:0x10039b650
@uri="druby://fe80::21b:63ff:fec9:335f%en1:57416", @ref=2149388540>, "I like
to say hi!"]Clicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
TupleSpaces
- Shared object space
- Atomic access
- Just like bulletin board
- Tuple template is
- [:name, :Class, object, ‘description’ ]
5 Basic Operations
- write
- read
- take (Atomic Read+Delete)
- read_all
- notify (Callback for write/take/delete)
Starfish
- Starfish is a utility to make distributed programming ridiculously easy
- It runs both the server and the client in infinite loops
- MapReduce with ActiveRecode or Files
starfish foo.rb
foo.rb
class Foo
attr_reader :i
def initialize
@i = 0
end
def inc
logger.info "YAY it incremented by 1 up to #{@i}"
@i += 1
end
end
server :log => "foo.log" do |object|
object = Foo.new
end
client do |object|
object.inc
endClicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
Starfish server example
ARGV.unshift('server.rb')
require 'rubygems'
require 'starfish'
class HelloWorld
def say_hi
'Hi There'
end
end
Starfish.server = lambda do |object|
object = HelloWorld.new
end
Starfish.new('hello_world').serverClicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
starfish client example
ARGV.unshift('client.rb')
require 'rubygems'
require 'starfish'
Starfish.client = lambda do |object|
puts object.say_hi
exit(0) # exit program immediately
end
Starfish.new('hello_world').clientClicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
starfish client example (another way)
ARGV.unshift('server.rb')
require 'rubygems'
require 'starfish'
catch(:halt) do
Starfish.client = lambda do
|object|
puts object.say_hi
throw :halt
end
Starfish.new
('hello_world').client
end
puts "bye bye"Clicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
MapReduce
- Introduced by Google to support distributed computing on large data sets on clusters of computers.
- Inspired by map and reduce functions commonly used in functional programming.
Starfish Server Example
ARGV.unshift('server.rb')
require 'rubygems'
require 'starfish'
Starfish.server = lambda{ |map_reduce|
map_reduce.type = File
map_reduce.input = "/var/log/apache2/access.log"
map_reduce.queue_size = 10
map_reduce.lines_per_client = 5
map_reduce.rescan_when_complete = false
}
Starfish.new('log_server').serverClicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
starfish client example
ARGV.unshift('client.rb')
require 'rubygems'
require 'starfish'
Starfish.client = lambda { |logs|
logs.each do |log|
puts "Processing #{log}"
sleep(1)
end
}
Starfish.new("log_server").clientClicking "Copy Code" button will copy the code into the clipboard - memory. Please paste(Ctrl+V) it in your destination. The code will get pasted. Happy coding from Wikitechy - ruby on rails tutorial - rails guides - ruby rails - rubyonrails - learn ruby on rails - team
Other implementations
- Skynet
- Use TupleSpace or MySQL as message queue
- Include an extension for ActiveRecord
- MRToolkit based on Hadoop
MagLev VM
- A fast, stable, Ruby implementation with integrated object persistence and distributed shared cache.