Second Extended Filesystem (ext2) and other improvements (#33)

Second Extended Filesystem (ext2) and other improvements (#33)

* Add a wrapper layer to vfs interface with intension to serve an unified API
  for file-system driver implementation and thus makes the flow clear.
* Localise iso9660's header file
* Make qemu launch cmd out of makefile to enable better flexibility
* remove an out-dated FIXME annotation

* Add ext2 protocol definitions and v_superblock construction

* Sparse Caching: cache framework for caching spatial data unit (#34)

* Add framework for sparse caching facility
Added a spinlock as a future-proof placeholder

* * Add bcache_zone to allow more fine gran control over lru zone used
  when creating a bcache
* Add lock on entire bcache instance to avoid concurrent modification
  on internal data structure
* Add missing free function for lru zone, introduce async freeing when
  a zone is too busy to be freed onsite
* Add interface for destroying spinlock when a guarded object is about
  to be freed, thus avoid locking-after-free.

* Refactor inode pcache to use bcache for better centralisation

* lru: add proper synchronisation ensurance to lru_zone
* lru: add twimap integration to all lru_zones
* lru: add extra statistical metrics
* change naming to spinlock relatives for better clarity
* twimap: go_next now called before read, to capture corner case of empty list
* clean-up and compliation fixes

* * blk_buf: buffering mechanism for blkio requests and their payloads
* Replace locking on each bcache node with ref counter, thus avoid
  creating weired locking pattern and potentially dead lock.
* add documentation on the stages that owloysius will
  recognise
* housekeeping: remove not-needed file

* * ext2: parsing superblock and block descriptor table
* ext2: locating and reading inode
* fsapi: add new helper method for initializing inode
* fsapi: add support to block buffering, allowing read/write
         of file system block to be efficient.

* * ext2: add support to read the inode data block and indirect link
  following.
* ext2: introduce btlb to accelerate indirect link following.
* blkbuf: add `refonce` to increament the internal ref counter for
  better cache management
* housekeeping: make the inline function at hash.h static
* housekeeping: rename fsapi_{get|put}blk to fsblock_{take|put}

* * Change the directory iteration pattern in vfs
  This allow us to add support of lseek functionality to directory. i.e.
  seek to i-th directory entry, similar to seekdir(2)

* * ext2: directory iteration and seeking.
* ext2: add a generic iterator for ext2 objects
* minor refactoring on some code snippet to reduce verbosity

* * ext2: read and read_page support on data blocks

* * proper error handling and reporting on blkbuf and ext2

* * vfs: fix an issue that async freeing of inode and dnode cachings
  could potentially result undefined access to an already freed
  v_superblock object. By tracking the no. of references of vsb and
  free it when and only when nobody is reference it.
* vfs: remove the auto-handling of . and .. dir entry from the readdir
  interface. Thus makes it a file system driver's responsibility
* ext2: fix the incorrect index usage when addressing an ext2 directory
  entry.
* ext2: add strneq for bounded string comparision.
* minor refactoring and tidying.

* * make things compile
* iso9660: fix the issue when selecting name for the 00/01 dir ident

* inode::itype bit-field redefinition & bug fixes

* rework the itype bit-fields for better clarity.
* fix couple of bugs in ls that does not check errno of readdir syscall
* add a new directive '?' to debug_sh (sh) to check exit status of
  previous invocation

* ext2: properly functioned the directory listing

* fix strncpy behavior to be smarter
* corret the inode number offset
* provide v_mount struct before mounting stuff, which makes more sense
* fix issue of btrie tree retrive incorrect leaf when order is a odd number
* fix the itype conversion from ext2 to lunaix

* ext2: basic file read functioning

* fix incorrect return size calculation in ext2::read and pcache::read
* make ext2db_iter terminate when datablock reaching the size of file.
* fix that cat does not check on negative return value of read upon
  read(2) error.

* ext2: inode and data block allocation

* ext2: add support to allocate inode and data block
* ext2: add support to de-allocate inode and data block
* ext2: managing the ext2_gdesc with bcache
* ext2: rename some functions for conciseness
* usr/sh: QoL improvement by faking the PATH env var

* ext2: inode data blocks allocation

* ext2: unifying the indirect block walking
* ext2: data block allocation for inode

* ext2: mkdir, rmdir, generic dir_ent insert/delete

* ext2: add mkdir and rmdir
* ext2: add inode creation
* ext2: add ability to do file write
* vfs: add the missing destruct callback for dnode
* fix: add a null check on bbuf_t related operations

* ext2: symlink, mknod, link/unlink, sync and fixes

* ext2: add support to symlink get/set
* ext2: add support to inode creation
* ext2: add link and unlink
* ext2: directory renaming
* ext2: fsync and full fs syncing
* ext2: add proper un-mounting
* ext2: inode metadata update & sync
* vfs : some refactoring
* vfs : add O_TRUNC and fix O_APPEND behaviour
* vfs : update the fsize after write
* twimap: fix issue that it failed to read mapped content

* update LBuild with new files, fix some issue with LConfig

* fix issue where LConfigProvider::has_config does not capture the case
  of boolean typed config
* fix issue where CHeaderConfigProvider:: try to appened non-str typed
  value to the header file generation flow

* various fixes for merging the master branch

* fix: the incorrect use of clz on unsigned long data type
* ref: add 64bits variances of clzl, llog2. provide handy definition
       of most significant bit index of unsigned long and int.
* ref: uncaptialize the ILOG2
* fix: the kernel stack creation and copy algorithm, make it
  less hacky
* fix: assertion on thread state in pwake is too strong. if a
  thread is already awakened by other event or killed, just remove
  it from wait list
* fix: change the tag calculation in ext2::bTLB, so it can contain
       validity bit
* fix: a block buffer should ref again when inserting into bTLB
* fix: mixed use of size_t and unsigned int, this is not working
       on 64bits platform
* fix: boot disk prober automatically use last device it scaned
       when no blkdev can be identified as boot disk
* feat: qemu.py (autoqemu) will now skip the block device if the
        image is not found, so it provides a less disastrous
        fallback behavior
* fix: always vzalloc for the bltb buffer.

* fix logic error in ext2 file write & inode creation flow

* rename the fsblock_take to fsblock_get
* add the fsblock_take for doing refonce function
* add a test program that write random string into
  file

* fix issues in dirent creation and file write

* fix:  incorrect calaculation of rec_len
* fix:  confusion on inode number (1 based) and inode
        index (0 based)
* fix:  racing when one process creating file while
        other try to read that directory structure.
* fix:  pcache write must not enforce the fpos to be
        page aligned all the time, otherwise, page
        cache paddings will be written.
* fix:  autoqemu: smp and memory parameter not set.
* fix:  rng generator failed to populate whole buffer
        other than the beginning few bytes
* fix:  failsafe unwrapper should follow x86_64 convention.
* fix:  blkbuf: define null bbuf as not initialized.

* modified ext2 now passed the e2fsck validation

* fix:  sync the changes to superblock.
* fix:  correct the rec_len when the dirent is last of this
         data block
* fix:  (ahci) add sanity check on DMA buffer size
* fix:  i_blocks calculation should include the intermediate blocks
         allocated for building indrections.
* ref:  some minor changes to the fsapi_vsb_*
* feat: add a printk emulation for quick log printing without
         the need of explicit specify the LOG_MODULE
* fix:  skip the sync if a blkio_req is pending. previous version
         only skip it after the access flag is changed, which may
         lead to undesired behaviour on some cases.

* fix issue in ext2_mkdir

* fix:   break the pwait into two parts: add to wait list and
          the actual wait start. Thus to resolve the racing
          problem in the blkio by ensure that the active thread
          will always in the blkio_req's waiting list even when
          the completion interrupt arrive before we invoke the
          actual waiting.
* fix:  dirty page cache should be flushed before closing the
          file.
* fix:  make most system calls preemptive in kernel space, to
          increase the responsive time and avoid interrupt
          squashing for i/o intensive device.
* feat: add mkdir program for testing

* protect the blkio and blkbuf with mutex lock.

* ref:   separate the non-reentrant and reentrant (nested)
          mutex locking/unlocking.

* Improvements patch: Kernel preemptions (#42)

* added mechanism for checking any stalling in kernel space

* ref:    wrap the syscall dispatching into a proper C function.
* ref:    separate the sched_pass into two category: yield and preempt
* ref:    add a redirection checking stage prior to the context restore,
           which enable us to extend in the future (if needed)
* feat:   add the to more pwait variant, for indicate that the event
           waiting should be opted in stall checking
* feat:   add thread statistics on kernel/user transition

* fix: outdated kernel stack reclaiming corrupt the other threads

* feat:   add an extra config term to detect dead loop in kernel space.

* minor adjustment on formating

* add comments on explaining switch_signposting

* file deletion and add file resizing

* fix:    tested and fixed the file deletion.
* feat:   inode resizing for dynamically deallocate data block
          that is not used after shrinking.
* fix:    incorrect thread reference when checking stalled task
* feat:   test program for file deletion/unlinking.

* update the inode attrs when setting symlink

* ref:  change some comments
* fix:  add FSTYPE_PSEUDO to distinguish filesystem that is
        capable of device-less mounting
* feat: a wrapper for filesystem registration
* ref:  refactor currently avaliable file system registration
        code with aforementioned wrapper

* update readme

* * rewrite the ext2_rename, to make it right

* feat: add support to FILE_TYPE and LARGE_FILE feature.
* fix:  feature check on mounting.
* fix:  memory leakage on ext2_dnode.
* fix:  double free on ext2_dnode when ext2_mkdir fail.
* fix:  properly release the ext2_dnode after ext2dr_remove.
* ref:  code clean up on ext2.