This web site contains complete details and all components of a complete
clinical
trials reporting engine that uses SAS®
software. Spectre (Clinical) is composed of two parts: Spectre Reporting
Macros and the Spectre System (the latter composed of both SAS
macros and shell scripts). Spectre also has a utility library of
macros that serves both these parts. The Reporting macros do both
safety
and efficacy reporting and run independently of the Spectre
System. You can run the Reporting macros to create reports even if the
Spectre System is not present. The Spectre System is a particular
method for providing titles and footnotes to the Reporting macros, running
reports and collecting together the output. If you provide your own titles
and footnotes by an alternate method and run the programs and collect the
output by an alternate method then this will work just fine with only the
Reporting
Macros part of Spectre. At the date of writing this, the Spectre
System is not in use anywhere in the world whereas the Spectre Reporting
Macros are.
The Spectre System is largely historical and remains there to
give you an idea as to how to set up a titles and footnotes system for
reporting macros and how to run suites of programs to create analysis datasets
and to create the output reports and gather them together at the end. It
was originally written to run on a Linux
or
Unix platform
but has since been ported over to Windows. Neither the original Unix system
nor the Windows version are known to be in use at the time of writing.
Both Spectre Reporting Macros and the Spectre System are what is know
as "back end" systems. It does the workbehind the scenes
of
a "front end" interface.
You will see the term "reporting engine" used, rather than the
more familiar "reporting system", because what is found on this web site
comprises the "core" of a reporting system and the engine underlying it.
You could, perhaps, take an existing reporting system and replace its core
macros with what is found here, without affecting the look and feel of
the existing reporting system, just like you might be able to replace the
engine of an automobile with an alternate engine.
If you have limited time and want the briefest possible summary of
the Spectre reporting system then use the link below. Spectre in a Nutshell
(If you landed on this page doing general web browsing and are not
interested in clinical reporting systems then you might like to look at
the sas macro page here. If
you are looking for clinical reporting macros then there are two
macros webbed here named %unistats
and %npcttab. %unistats has
its own Powerpoint slide show
here.
There is a lot of practical Unix stuff that covers both Unix commands and
shell scripting that you can link to here
and there are a lot of shell script examples you can link to here.
Thanks for your visit)
Disclaimer
All software comes with a disclaimer and the same is true for both parts
of Spectre. When you install software you have to agree to the terms of
using it to the effect that you should ensure the software is suitable
for your purposes and that no liability is excepted if the software goes
wrong. You are now expected to use the Spectre System - just the Reporting
Macros. These macros are very stable but bugs are found in them from time
to time. Also, you should be aware of what the macros are doing and be
particularly careful abiout the statistical values you might choose to
show that come out of these reporting macros. Check these values. If the
value looks wrong then the chances it is wrong and this is likely
due to it rerining the values in a way that you are now expecting. It is
up to you to ensure what is being reported at least makes sense and looks
reasonable. So first, an important disclaimer, and that is....
DO NOT ASSUME THE CODE ON THIS WEB SITE IS "VALIDATED"
FOR YOUR PURPOSES
If you intend to use this code for clinical reporting purposes, rather
than just for education, then you can only treat what you find here
as a validated system if you have validated it yourself, in accordance
with industry regulations and your own Standard
Operating Procedures. All of the reporting macros have been validated
by one of the known users but they have validated it for their own particular
mode of use. This does not make it validated for your purposes.
Although I have a great deal of confidence in the code members webbed here,
having used them for years, and knowing that most code members run live,
you should not assume, without thorough checking, that they will work for
you in the way you want or be suitable for your needs. To stress this point,
every single code member will carry a disclaimer in the header to remind
you of this. No code members should be used unless you fully understand
and accept the disclaimer in the header.
Download
You can download the Spectre system from the page you can link to below
(the "download" page) as a number of zip files. You will probably just
be interested in the clinical reporting macros but these also rely
on a library of macros that are the utility macros so if you are
only interested in the Clinical Reporting macros then you must download
the utility macros as well. You need to make both libraries available
and place both libraries on your sasautos path for the Clinical
reporting macros to work.
Spectre is based on an old Unix-based clinical reporting system that was
once used at Glaxo Wellcome before they merged with SmithKline Beecham
to form GSK. This old clinical reporting system was phased out after the
merger and replaced with a new system. I was working at the company at
the time. I preferred the older system because, by accident and not design,
it was structured in a way that it was just a few steps short of achieving
the ultimate goal of a reporting system, which is to be a "push button"
reporting system (explained later). Because I liked this old system I rewrote
it, simplifying and redesigning it, to put it in a better position to be
a very high performance reporting system and possibly achieve the status
of being a "push button" system.
The components of a "reporting engine"
For me, at least, a "reporting engine" has a few identifiable components.
On the reporting side, it should have the ability to handle the
bulk of safety reporting, and that requires two major macros - one
macro to calculate summary statistics and category counts with percentages
and another macro to calculate unique patient counts and percentages
(and optionally event counts). Most large pharmaceutical companies (and
many smaller ones and CROs) have such macros (Spectre Reporting goes beyond
this need by being able to do efficacy reporting as well).
On the system side, a "titles" (and footnotes) system is
required such that titles and footnotes can be stored outside sas code
members and somehow get included into tables and listings.
A means of running all the programs in the correct order to create
the output.
A means of gathering all the outputs produced in the correct order
or at least telling a separate documentation system about it.
There are many things that could be added to the above list that people
would think are essential but if they have these components in place then
I would say they have a "reporting engine" and the "core" of a reporting
system. A reporting "system" will have a fuller set of macros for
producing reports and some of these macros will call one of the two core
macros described above. Quite often, reporting systems have sophisticated,
web-based front ends. There are no current plans to make a front end available
as yet.
Why you need a reporting system
All pharmaceutical companies want to be efficient in producing clinical
reports to save on costs. For the large pharmaceutical companies, perhaps
with hundreds of programmers worldwide, they know that they can save a
lot of programming time by having "standard macros" to do most of the reporting.
Maybe 25% or more of programmer time can be saved. That comes to a lot
of potential money saved, so the larger pharmaceutical companies will invest
whatever it takes to build a reporting system. And if they have enough
programmers to select from, they can find the skilled people to achieve
this. It is not so easy for smaller companies or CROs
to create such a reporting system as they have fewer skilled programmers
to choose from and whatever investment they make to build such a system
will be a disproportionally higher proportion of their programming costs.
There are other reasons for having such a reporting system. Reporting
systems can help with presentation in that they impose a consistency
in layout style. They can help with reliability and accuracy of
safety reporting using the two major reporting macros as described above,
since these macros, through their thorough usage and careful monitoring
and maintenance, will arrive at the point where they are completely trustworthy
and so their output will require only minimal checking. A good system will
ensure
all programs are run in the correct order and that a record
is kept of who ran what and when. Also, that a list of outputs created
is recorded somewhere.
The ultimate goal of a reporting system
The ultimate goal of a clinical reporting system is to produce reports
automatically from clinical data. Now if you thought that last sentence
was clear, I'm going to disappoint you by qualifying what I mean by "clinical
data". This "clinical data" might not be the same as the data collected
for a clinical trial. The structure might have changed as well as the naming
conventions and other values might have been derived and merged in with
the data to make "derived datasets" or "reporting-ready datasets"
and it is that to which I refer when I write "clinical data". If this data
exists in a standard form (which might be the case in the future as a result
of the CDISC project)
and the type of analysis is decided in advance, then it should be possible
to generate reports automatically - just by pushing a button, maybe. Achieving
this might take many years from the date of my writing this and indeed,
it may never be achieved, but if this goal is constantly striven for, greater
efficiencies will be realized along the way. Spectre embraces this ultimate
goal of being a "push-button" reporting system. Short calls to reporting
macros can be held in the same place that titles and footnotes are defined
and that can lay the foundation for an entirely automated reporting system.
It has already been used in this way to generate all the tables for one
clinical study except it fell short in that the macros it called to produce
the reports were study specific, since the data structure and variable
names varied from study to study. But when the data structures are harmonized
then that ultimate goal should be achieved.
Spectre "Extras"
When Spectre was first implemented, the company was entering into a merger.
It was a requirement that the reporting system could produce output in
the current style as well as a future style (yet to be defined) and that
both styles of reports would be recreatable. So the system had to have
the
capability of producing output in more than one client style. Though
not written with CROs
in mind, this will be of obvious benefit to them since they work for multiple
clients. It also makes Spectre a candidate for becoming a "standard" reporting
engine for use across the industry.
Safety vs. Efficacy Reporting
The Spectre Reporting Macros do both safety and efficacy reporting
in the same report in the way you specify. The reporting macros have powerful
statistical capabilities and huge flexibility to allow you to include efficacy
statistics into the report from whatever source and in any position you
specify. You just have to know the "proc glm" or "proc freq" options to
specify, what ods tables are created as a result and what variables in
those ods tables to use and you can place them whereever you want without
need for further programming. For simple p-values, it is a lot easier.
The User Guides
I hope you found the above introduction to Spectre interesting. Hopefully,
you will have caught a glimpse that this is a reporting engine with purpose
and direction. The next step is to read the User Guides that can be linked
to below. Remember that the Reporting Macros are independent of the System
macros. I have flagged the "system" topics below so that you can
skip over them if you want to. You should read the pages in the order listed.
If you are a programmer then you will not need to know about Spectre administration.
The administrator has extra tasks to do. Before you start working on a
study, the administrator will have set up the file "protocol.txt" in your
programs folder and they will have put the file "titles.template" there
which you can copy and use as the template for your titles members. They
will likely be the person to run the suite of programs and to create the
PDFs using the utilities provided. If the suite of programs is large and
the amount of output large then the administrator will have to split the
file "donelist.txt" (the list of all output produced) into "donelist sections"
and to create a PDF from each section. And when the PDFs have been created,
they will be the person to run the "printpdfbookmarks" utility to extract
the list of PDF bookmarks from the PDF files. If Spectre is being used
to create the derived datasets then the administrator will check the naming
convention of the programs to ensure that the suite of programs runs in
the correct order and to ask programmers to rename some of the programs
if this is not the case. They will likely check the naming convention used
for the programs that produce output and sometimes programmers will be
asked to rename their programs (there are utilities to help with this described
on the "Utility Scripts" page). So if you are a programmer, read the administrators
guide if you are interested, but you will find more to interest you in
the "Scripts and Macros" section.
If you are a programmer using Spectre then after a while you will understand
that it is aimed at increasing speed and efficiency. It is there to remove
barriers
to greater productivity, not to add them. Greater speed and efficiency
means that single programmers will maybe end up with a hundred programs
or more. But having a hundred programs or more has the potential of turning
into an administrative nightmare. It would be like winning the technical
battle of efficient coding only to be overwhelmed by sheer weight of numbers.
That would be very stressful for a programmer but help is at hand. Spectre
has many utility scripts to help the programmer manage his hundred or more
programs written using SAS software and it is important for the programmer
to be aware of these scripts and how they can help. Without knowledge of
these scripts, a programmer will eventually hit the barrier caused by sheer
weight of numbers. But knowing that these scripts exist and knowing how
to use them, having hundreds of programs should not be a problem. It will
take almost none of your time and concentration to deal with it.
Here is a typical example of having to divert your attention to work
on one of your hundreds of programs and how you would cope with it.
Suppose someone tells you that they would like the footnote to be changed
for
Table 13.4.5.1 in study XYZ. There is no need to look
up this table in a spreadsheet somewhere. You go to one of your terminal
windows, you probably have variables set up in your ".bashrc" member (sometimes
it is called ".bashrc_own") to help you jump to directories. So you type
in the command "cd $xyzp" (or something like that) to take you to
the study XYZ programs directory. You then type in the command "wt 13.4.5.1"
to tell you what program it is. You then set "prog" equal to this
program and then (assuming "prog" has already been "exported") you type
in the command "titles" and you have the titles member opened in
edit mode. You then change the footnote, recreate the titles members with
the "crtitlesds" command (you may even have an alias for this that
is shorter. I use "crt" as an alias for this). Then when this completes
you type in "sasb". You see that there are no errors or warnings
so you type in "lis" to look at the output. You see that the new
footnote is fine so you close the window and reset the current directory
to what it was before using the command "cd -", and it is done.
You are back working on what you were doing before after only a few seconds
have passed. You hardly noticed the intrusion and now you have even forgotten
about it.
If you know these scripts well and are familiar with Unix commands then
an interruption, such as the one described above, will not break your concentration
and you will find it easy to cope with hundreds of programs. As for writing
programs using SAS software, you will find many macros available to you,
linked to from these pages, to do common difficult tasks. They will help
you to write portable macros that can be used in multiple studies. After
a while, you will find that you are writing less and less code and instead,
copying macros and titles from study to study. There are scripts to help
you do this. After you copy them across, there are scripts to fix the study,
drug and increment names in the headers. There are scripts to help you
do a mass renaming of files. There are scripts to help you tidy up your
".log" and ".lst" files that get left over after a mass renaming has been
done. If you take the trouble to learn about these scripts and macros then
you will easily be able to cope with having hundreds of programs. Hopefully,
you will you become one of the most highly productive clinical reporting
programmers in the field and yet be stress free with it.
The most important macros are %titles, %openrep, %closerep, %popfmt,
%unistats and %npcttab. These have their own pages that you can link to
above and you should make sure you read them thoroughly. The "macros" page
you can link to below just contains summary information about these important
macros. As for the scripts, the ones the programmer will be most interested
in are the "Utility Scripts", as those are the ones that will help speed
and organize their work.
It is important to realise that Spectre is a very simple reporting system.
It is minimilistic and intended that way. It makes available a huge amount
of functionality (particularly with the reporting macros) without expanding
its role and forcing you to change the way you work with it. Instead, it
works for you. Used as a back-end system in the way described here, it
is likely far simpler than any other reporting system currently in use
at any major pharmaceutical company. Many front-end systems become over-complicated,
difficult to use and slow over time, hugely reducing productivity. But
Spectre will always remain as as a powerful, efficient and fast back-end
system. It has a lot of components, as you will see if you look, but then
so does any motorised utility that does something simple like help you
cut your grass. Using Spectre is logical and simple. To help you realise
that, you can link to the page below and see a very simple dummy study,
with just two derived (dummy) datasets and two tables, reported from empty
directories through to final report run, showing all the administrative
tasks performed. Finally, the transformation to get it ready for a "push
buttion" reporting system is demonstrated and explained. If you wanted
to skip through it quickly, it might only take you a few minutes.
I have included three other topics with their own macros that were not
part of the original Spectre as I am sure you will have a use for them
occasionally.
In creating Spectre, I have used some techniques with sas that not all
sas programmers will be familiar with. Without using these techniques,
I could never have written Spectre in the time I did. All the techniques,
as they relate to writing a reporting system, will be explained in the
following document you can link to.
As programmers using Spectre become more proficient, they can find that
they are responsible for hundreds of programs at a time. Dealing with this
many files is easier if you have a good knowledge of Unix commands and
you know how to combine them. This is a higher level of skill than most
sas programmers would have who work on a Unix platform so there are documents
on this web site to help you increase your Unix skills.
If you just want to use the %npcttab and %unistats macros but do not
want to set up the whole of Spectre (Clinical) then installing it is easy.
All you need to do is download the clinmacros and utilmacros
and put them in their own libraries defined to the sasautos path. The rest
of the information you can read here is to do with a full Spectre System
install for those users planning to set up a full clinical reporting system.
Spectre will work best on a Linux or Unix platform with a proper Linux
or Unix version of SAS software (i.e. supporting the -stdio option).
Although it has been ported over to Windows you will have to use the Cygwin
Linux emulator to run the Spectre scripts. On a Windows platform the scripts
to convert output to Postscript files and thence to PDFs will not work
as to get this working would require a lot of disk space and a lot of work
that is probably not needed. If you are a CRO then it is unlikely the client
will ask for your output in PDF format. If they want it in that format
then it is easier if the client accepts your text output files as deliverables
and does the conversion to a PDF themselves. Apart from Postscript files
and PDFs, the rest will work on a Windows platform with Cygwin installed.
It will work fine with SAS Learning Edition 4.1 buth this will limit the
number of observations you can read so you can not use them for production
work.
The full installation of Spectre (Clinical) is quite complex so there
are guides for this that you can link to below.
Spectre System has only a few key components. So long as those components
are not changed, there is huge scope to tailor it so that it works in the
way you want it to. I do not want to give the impression that Spectre System
is a fixed system. It is not. What you see on this web site is more a full-blown
example of what it can do. You can change Spectre System in many ways to
suit, once you have a feel for what these key components are. This is explained
on the following page.
Once you know the key components of Spectre System it should put you in
a position of being able to amend Spectre to suit your site standards.
This is explained on the following page.
The Spectre Clinical reporting macros require a licence for use as of 2012.
The rest is unsupported and free. The licence fee is described on the download
page. Further details are on the "terms" page you can link to below.
The philosophy of the licence fees is that those who contribute to Spectre
are rewarded with the best deals. The Alpha site uses the macros for free
indefinitely because they contributed to their development. The reason
the macros are reliable is largely due to their work. They are a small
CRO. The next few customers hoped for will be very large CROs and they
will have to do a lot of work, along with myself, to bring the product
up to a high standard. They will be rewarded for their efforts by the low
initial licence fee maintained indefinitely for their licence block count.
When the clinical reporting macros have finally achieved something near
their finished state then the licence fee will be increased for new users
to a commercial level, with annual increases to cover inflation and extra
costs.
Everything you need to know about Spectre should be documented on this
web site. A few unrelated topics are included in the FAQ which you can
link to below. I hope you will find Spectre fast to learn, easy to use
and that it will increase productivity several fold.
SAS and all other SAS Institute Inc. product or service names are registered
trademarks or trademarks of SAS Institute Inc. in the USA and other countries.
® indicates USA registration.