FlashGraph-ng
A new frontier in large-scale graph analysis and data mining
 All Classes Namespaces Functions Variables Typedefs Enumerations Enumerator Pages
Classes | Typedefs | Enumerations | Functions | Variables
safs Namespace Reference

Classes

class  page_byte_array
 
class  comp_io_scheduler
 
class  direct_comp_io
 
class  callback
 
class  io_status
 
class  io_interface
 
class  io_select
 
class  comp_io_sched_creator
 
class  file_io_factory
 
class  data_loc_t
 
class  request_range
 
class  user_compute
 
class  compute_allocator
 
class  io_request
 
class  memory_manager
 

Typedefs

typedef int file_id_t
 

Enumerations

enum  { , REMOTE_ACCESS, GLOBAL_CACHE_ACCESS, PART_GLOBAL_ACCESS, DIRECT_COMP_ACCESS }
 

Functions

io_interface::ptr create_io (std::shared_ptr< file_io_factory > factory, thread *t)
 
file_io_factory::shared_ptr create_io_factory (const std::string &file_name, const int access_option)
 
void init_io_system (config_map::ptr map, bool with_cache=true)
 
void destroy_io_system ()
 
bool is_safs_init ()
 
const RAID_config & get_sys_RAID_conf ()
 
const std::vector< int > & get_io_cpus ()
 
void print_io_summary ()
 
void set_file_weight (const std::string &file_name, int weight)
 

Variables

const int INVALID_FILE_ID = -1
 
const int IO_QUEUE_SIZE = 10
 
const int MAX_NUM_FLUSHES_PER_FILE = 2048
 
const int NUM_REQS_BY_USER = 1000
 
const int MIN_NUM_ALLOC_IOVECS = 16
 

Detailed Description

Copyright 2014 Open Connectome Project (http://openconnecto.me) Written by Da Zheng (zheng.nosp@m.da19.nosp@m.36@gm.nosp@m.ail..nosp@m.com)

This file is part of SAFSlib.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Typedef Documentation

typedef int safs::file_id_t

The data type for a SAFS file identifier.

Enumeration Type Documentation

anonymous enum

This defines the method of accessing a SAFS file. There are six options.

Enumerator
REMOTE_ACCESS 

This method is equivalent to Linux direct I/O. It accesses data in a SAFS file without caching. It only supports asynchronous I/O.

GLOBAL_CACHE_ACCESS 

This method is equivalent to Linux buffered I/O. It accesses data in a SAFS file with caching. It supports both synchronous I/O and asynchronous I/O.

PART_GLOBAL_ACCESS 

This method also accesses a SAFS file with caching. It localizes data access in the page cache. The implementation of this method is incomplete right now, so it shouldn't be used.

DIRECT_COMP_ACCESS 

This method accesses a SAFS file with asynchronous user-task interface, but without page cache.

Function Documentation

io_interface::ptr safs::create_io ( std::shared_ptr< file_io_factory >  factory,
thread *  t 
)

This function creates an I/O instance from the I/O factory.

Parameters
factorythe I/O factory for creating an I/O instance.
Returns
an I/O instance.
file_io_factory::shared_ptr safs::create_io_factory ( const std::string &  file_name,
const int  access_option 
)

This function creates an I/O factory of the specified I/O method.

Parameters
file_namethe SAFS file accessed by the I/O factory.
access_optionthe I/O method of accessing the SAFS file. The I/O method can be one of REMOTE_ACCESS, GLOBAL_CACHE_ACCESS and PART_GLOBAL_ACCESS.
void safs::destroy_io_system ( )

This function destroys SAFS. It should be called at the end of a program.

const std::vector<int>& safs::get_io_cpus ( )

Get the CPU cores that are used for I/O dedicatedly.

const RAID_config& safs::get_sys_RAID_conf ( )

This function gets the RAID configuration of SAFS.

void safs::init_io_system ( config_map::ptr  map,
bool  with_cache = true 
)

This function initializes SAFS. It should be called at the beginning of a program. It can be invoked multiple times. If it is executed multiple time successfully, destroy_io_system() needs to be invoked the same number of times to complete clean up the I/O system.

Parameters
mapthe SAFS configuration.
with_cachedetermine whether the I/O system is initialized with page cache.
bool safs::is_safs_init ( )

This function tells whether SAFS is initialized successfully.

void safs::print_io_summary ( )

This function prints the summary info on I/O statistics in the system.

void safs::set_file_weight ( const std::string &  file_name,
int  weight 
)

The users can set the weight of a file. The file weight is used by the page cache. The file with a higher weight can have its data in the page cache longer. This function isn't thread-safe and should be used before I/O instances are created.

Parameters
file_nameThe file name.
weightThe new weight of the file.

Variable Documentation

const int safs::INVALID_FILE_ID = -1

Invalid SAFS file identifier.

const int safs::IO_QUEUE_SIZE = 10

The size of a high-priority I/O queue. It's in the number of I/O messages.

const int safs::MAX_NUM_FLUSHES_PER_FILE = 2048

This number defines the max number of flushes sent to an SSD. The number is set based on the performance result. It varies in different SSDs. It's better that we can estimate at runtime.

const int safs::MIN_NUM_ALLOC_IOVECS = 16

The min size of IO vector allocated for an IO request..

const int safs::NUM_REQS_BY_USER = 1000

The number of requests issued by user applications in one access.