Chapel Evolution¶
Like any language, Chapel has changed over time. This page is designed to describe significant language changes that have a high likelihood of breaking existing user codes or code samples from presentations or papers that predated the changes.
Note that the compiler flag --warn-unstable
is available and can be
useful when migrating programs to the current version of the language.
The purpose of this flag is to identify portions of a program that use a
language or library feature has recently changed meaning or which is
expected to change meaning in the future.
version 2.0, March 2024¶
Default task intents for arrays¶
In 2.0, the default task intent for an array is now determined by the outer
variable. If the outer array is const
then the default intent is const
,
otherwise the default intent is ref
. Therefore, if an array is modifiable
outside a parallel block, it is modifiable inside the parallel block. It is no
longer necessary to use an explicit intent like with (ref myArray)
to
modify myArray
in a parallel block. This change applies to forall
,
coforall
, begin
, and cobegin
.
Consider the following code which illustrates this.
proc myFunction(ref A: []) {
begin {
A = 17;
}
}
The default task intent for A
is ref
, since the argument formal A
is mutable. This simplifies parallel code, making it simpler and cleaner to
write.
Prior to 2.0, the above begin
would have resulted in a deprecation
warning. In 2.0, this is valid code again.
Associative Domains default to parSafe=false
¶
Associative domains have been stabilized and prioritize performance by default; however, some diligence is is required for correct use..
Associative domains in Chapel have a parSafe
setting that
determines their behavior when operated on by concurrent tasks.
parSafe
stands for “parallel safety”. Setting
parSafe=true
allows multiple tasks to modify
an associative domain’s index set concurrently without race conditions.
It is important to note that parSafe=true
does not protect the
user against all race conditions. For example, iterating over an associative
domain while another task modifies it represents a race condition and the
behavior is undefined.
See the documentation
for more information.
The default of parSafe=true
added overhead to operations and made
programs slower by default, even when such safety guarantees were not needed.
This is because it uses locking on the underlying data structure each time the
domain is modified. This overhead is unnecessary, for example, when the domain
is operated upon by a single task.
Motivated by this we have changed their default from
parSafe=true
to parSafe=false
.
With this change associative domains have been stabilized, except for domains
requesting parSafe=true
, which remain unstable.
Here’s a breakdown of the changes and how they might impact your programs:
- New default for associative domains:
Previously, associative domains were “parSafe” by default. This has changed to
parSafe=false
. For example:var dom: domain(int);
used to imply that
dom
was set toparSafe=true
but now it defaults toparSafe=false
.This means that the checks to guarantee parallel safety are no longer inserted by default, thus improving performance. Therefore, it is now the user’s responsibility to ensure parallel safety as needed.
A warning will be generated for domains without an explicit
parSafe
setting, to draw user attention to code that may need to be updated, unless compiled with-s noParSafeWarning
.var d1: domain(int); / warns var d2: domain(int, parSafe=false); / does not warn
where the compilation output of the above program would look as follows:
$ chpl foo.chpl foo.chpl:1: warning: The default parSafe mode for associative domains and arrays (like 'd1') is changing from 'true' to 'false'. foo.chpl:1: note: To suppress this warning you can make your domain const, use an explicit parSafe argument (ex: domain(int, parSafe=false)), or compile with '-snoParSafeWarning'. foo.chpl:1: note: To use the old default of parSafe=true, compile with '-sassocParSafeDefault=true'.
Since
const
domains are never modified, they are exempt from these warnings as changing their default toparSafe=false
does not have the potential to impact correctness.const dom: domain(int); / does not warn
In order to ease the transition, users can temporarily revert to the old behavior by compiling with
-s assocParSafeDefault=true
.$ chpl defaultAssociativeDomain.chpl -s assocParSafeDefault=true
If a program used associative domains and relied on
parSafe=true
, it might be useful to try compiling with-s assocParSafeDefault=true
and then add an explicitparSafe
argument for each associative domain individually, to ensure no races are introduced into the program by forgoing the parallel safety guarantees.parSafe=true
domains are unstable:Domains using
parSafe=true
are still considered unstable and continue to trigger unstable warnings when declared. For example:var dom: domain(int, parSafe=true); / generates unstable warning
generates the following compilation output:
$ chpl bar.chpl --warn-unstable bar.chpl:1: warning: parSafe=true is unstable for associative domains
- Associative domain literals:
Associative domain literals also generate warnings by default. Use explicit type declarations like
domain(int, parSafe=false)
to avoid them.var d1 = {"Mon", "Tue", "Wed"}; / warns var d2: domain(string, parSafe=false) = {"Mon", "Tue", "Wed"}; / does not warn
where the compilation output of the above program would look as follows:
$ chpl baz.chpl baz.chpl:1: warning: The default parSafe mode for associative domains and arrays (like 'd1') is changing from 'true' to 'false'. baz.chpl:1: note: To suppress this warning you can make your domain const, use an explicit parSafe argument (ex: domain(int, parSafe=false)), or compile with '-snoParSafeWarning'. baz.chpl:1: note: To use the old default of parSafe=true, compile with '-sassocParSafeDefault=true'.
version 1.32, September 2023¶
c_string
deprecation¶Version 1.32 deprecates the
c_string
type in user interfaces. Please replace occurrences ofc_string
withc_ptrConst(c_char)
. Note that you need touse
orimport
theCTypes
module to have access toc_ptrConst
andc_char
types.Here are some cases where directly replacing
c_string
withc_ptrConst(c_char)
may not work and what to do instead:if your code is…
update it to…
casting
c_string
tostring
use a
string.create*Buffer()
methodcasting
c_string
tobytes
use a
bytes.create*Buffer()
methodcasting
c_string
to other typecreate a string and cast it to other type
casting
string
toc_string
replace cast with
.c_str()
casting
bytes
toc_string
replace cast with
.c_str()
casting other type to
c_string
create a string and call
.c_str()
on itusing
param c_string
use
param string
Additionally, several
c_string
methods are deprecated without replacement:.writeThis()
.serialize()
.readThis()
.indexOf()
.substring()
.size
*
An equivalent for
.size
is the unstable procedurestrLen(x)
in theCTypes
module.The default intent for arrays and records¶
In version 1.32, arrays and records now always have a default intent of
const
. This means that if arrays and records are modified inside of a function, acoforall
, abegin
, orcobegin
, they must use aref
intent. This also means that record methods which modify their implicitthis
argument must also use aref
intent. Previously, the compiler would treat these types as eitherconst ref
intent orref
intent, depending on if they were modified. This change was motivated by improving the consistency across types and making potential problems more apparent.Since there is a lot of user code relying on modifying an outer array, the corresponding change for
forall
is still under discussion. As a result, it will not warn by default, but modifying an outer array from aforall
might not be allowed in the future in some or all cases.Consider the following code segment, which contains a
coforall
statement which modifies local variables. Prior to version 1.32, this code compiled and worked without warning.var myInt: int; const myDomain = {1..10}; var myArray: [myDomain] int; coforall i in 2..9 with (ref myInt) { myInt += i; myArray[i] = myArray[i-1] + 1; }
Note that to modify
myInt
, an explicitref
intent must be used whereasmyArray
can be modified freely. The changes to the default intent for arrays is an attempt to remove this inconsistency and make the treatment of types in Chapel more uniform. This code also modifies bothmyInt
andmyArray
in a way that can produce race conditions. WithmyInt
, it is very apparent that there is something different than a simple serial iteration occurring and this can signal to users to more careful inspect their code for potential bugs. HowevermyArray
can be used without that same restriction, which can be a source of subtle bugs. In 1.32, the loop is written as:var myInt: int; const myDomain = {1..10}; var myArray: [myDomain] int; coforall i in 2..9 with (ref myInt, ref myArray) { myInt += i; myArray[i] = myArray[i-1] + 1; }
This removes the inconsistency and calls greater attention to potential race conditions.
This change also applies to procedures. Consider the following procedure:
proc computeAndPrint(ref myInt: int, myArray: []) { ... }
It is clear that
myInt
may be modified and a user of this function can save this value beforehand if they need the value later. But without knowing what is contained in this function, it is impossible to tell ifmyArray
is going to be modified. Making the default intent for arraysconst
removes this ambiguity.This consistency is extended to records as well. Consider the following record definition:
record myRecord { var x: int; proc doSomething() { ... } }
Without knowing what the body of
doSomething
does, it is not clear whetherx
may be modified. In version 1.32, ifx
is modified the method must be marked as a modifying record using a this-intent.record myRecord { var x: int; proc ref doSomething() { ... } }
Now it is clear that the method may modify
x
.version 1.31, June 2023¶
Version 1.31 renames and adjusts two of range’s parameters, formerly
range.boundedType
andrange.stridable
, as well as the former domain parameterdomain.stridable
. For details please see Range Types in the online documentation for Version 1.31.Range boundedType / bounds parameter¶
Prior to Version 1.31, the boundedness of a range
r
was determined byr.boundedType
. As of 1.31, it is determined byr.bounds
. At the same time, the type of this field changed from:enum BoundedRangeType { bounded, boundedLow, boundedHigh, boundedNone };
to:
enum boundKind { both, low, high, neither };
This change helps make Chapel code shorter, improving its readability.
When updating your code, simply update the names accordingly. For example, from:
if myRange.boundedType == BoundedRangeType.boundedLow then ....;
to:
if myRange.bounds == boundKind.low then ....;
Range and domain stridability / strides parameter¶
Prior to Version 1.31, ranges and domains had the parameter
stridable
, which was a boolean that indicated whether the given range or domain allowed non-unit strides. As of 1.31, this parameter is replaced withstrides
whose type is:enum strideKind { one, negOne, positive, negative, any };
This change creates additional opportunities for optimization, for example in the cases where the range’s stride is known at compile time to be positive or to be -1. This also avoids a terminology problem where
stridable=false
implied, incorrectly, that a range could not be strided. Thestrides
values are now self-explanatory instead of the non-specific valuestrue
andfalse
.When updating your code, update the field name and replace boolean values with enum values. For example:
change from…
to…
myRange.stridable
myRange.strides
if myRange.stridable then
if myRange.strides != strideKind.one then
range(stridable=false)
range(strides=strideKind.one)
range(stridable=true)
range(strides=strideKind.any)
another potential replacement:
range(strides=strideKind.positive)
When getting an error like “assigning to a range with boundKind.positive from a range with boundKind.any”, insert a cast to the desired range type. Analogous updates are needed in code operating on domains.
version 1.28, September 2022¶
Version 1.28 included some significant changes to the overload resolution rules. In addition, it enabled implicit conversion from
int(t)
touint(t)
. This section discusses some example programs that behave differently due to these changes.See also:
Behavior Differences for Mixes of Signed and Unsigned¶
Prior to 1.28, numeric operations applied to a mix of signed and unsigned types could have surprising results by moving the computation from a particular bit width to another—or by moving it from an integral computation to a floating point one.
For example:
var myInt:int = 1; var myUint:uint = 2; var myIntPlusUint = myInt + myUint; / what is the type of `myIntPlusUint`?
Before 1.28, this program would result in compilation error, due to an error overload of
operator +
in the standard library.Version 1.28 adds the ability for an
int
to implicitly convert touint
and removes the error overload. As a result, theuint
version ofoperator +
is chosen, which results inmyIntPlusUint
having typeuint
.This behavior can also extend to user-defined functions. Consider a function
plus
defined forint
,uint
, andreal
:proc plus(a: int, b: int) { return a + b; } proc plus(a: uint, b: uint) { return a + b; } proc plus(a: real, b: real) { return a + b; } var myInt:int = 1; var myUint:uint = 2; var myIntPlusUint = plus(myInt, myUint);
In 1.27 the call to
plus
would resolve to thereal
version becauseint
could not implicitly convert touint
, but bothint
anduint
could implicitly convert toreal(64)
. As a result,myIntPlusUint
had the typereal
. This change from integral types to floating point types could be very surprising.In 1.28 the call to
plus
resolves to theuint
version, andmyIntPlusUint
has typeuint
.This behavior also applies to
int
anduint
types with smaller widths:var myInt32:int(32) = 1; var myUint32:uint(32) = 2; var myInt32PlusUint32 = myInt32 + myUint32;
In 1.27, the
int(64)
+
operator is chosen (because bothint(32)
anduint(32)
can implicitly convert toint(64)
), which results inmyInt32PlusUint32
having typeint(64)
. This could be surprising when explicitly working with 32-bit numbers.In contrast, in 1.28, due to the ability for
int(32)
to implicitly convert touint(32)
, theuint(32)
version of the+
operator is chosen andmyInt32PlusUint32
has typeuint(32)
.Param Expression Behavior¶
Some expressions consisting of mixed-type literal or
param
values now have different behavior. For example:var x = 1:int(8) + 2; / what is the type of `x` ?
Note in this example that the literal
2
is aparam
with typeint(64)
and that1:int(8)
is aparam
with typeint(8)
.In 1.27, this program would output
int(8)
, because the overload resolution rules would favor the+
overload using the type of the non-default-sizedparam
. The result is that in 1.27,x
had typeint(8)
.In 1.28, the rules are simpler and a closer match to the corresponding case with regular variables (
myInt8 + myInt64
). There is no longer any special behavior for non-default-sizedparam
. As a result, the valuex
now has typeint(64)
.For similar reasons, the type of
nI
in the following code is nowint(64)
where previously it wasint(32)
:const nI = ((-2):int(32))**53;
A similar change can also appear with range literals that use mixed type
param
lower and upper bounds. The following range construction also makes use of the new implicit conversion fromint(t)
touint(t))
:var r8 = 1:int(8)..100:uint(8); writeln(r8.type:string);
In 1.27, this would generate a range with index type
int(16)
. In 1.28, it produces a range with index typeuint(8)
.Speaking of range literal construction, a range like
1:int(8)..10
still produces anint(8)
range in 1.28. However, as we have discussed, something like1:int(8) + 10
would result in anint(64)
. For now, the range implementation has been adjusted to preserve the old behavior specifically for the..
operator. However, this may change in a future release.Change for some mixed int/uint overloads¶
This example shows a change in behavior for two overloads where one is
int
and the other isuint
:proc dbm(a:int(8)) { writeln("dbm int8"); } proc dbm(a:uint(64)) { writeln("dbm uint64"); } dbm(42:int(64));
Previous to 1.28, this program would call the
int(8)
version of the function. It can do that because the compiler knows that theparam
value42
will fit into anint(8)
. Such a conversion is called aparam
narrowing conversion. However, in 1.28, this function now calls theuint(64)
version of the function. The main reason for this is that the 1.28 rules prefer to not doparam
narrowing conversion when another candidate does not need it. In this case,int
touint
is not aparam
narrowing conversion so that is preferred.Change for function visibility / shadowing¶
The new overload resolution rules in 1.28 consider function visibility or shadowing before considering how well the arguments match. Consider this example:
proc f(arg: int) { writeln("f int"); } proc main() { proc f(arg) { writeln("f generic"); } f(1); / which `f` does this call? }
Inside of
proc main
, the call tof
now resolves to the generic inner function. In contrast, in version 1.27, the outerproc f(arg: int)
would be called.version 1.22, April 2020¶
0- vs. 1-based Indexing¶
Version 1.22 makes a major breaking change to Chapel with respect to indexing for cases that involve implicit indices. Historically, Chapel has used 1-based indexing for such cases, where it now uses 0-based indexing.
The major types that are affected by this change are tuples, strings,
bytes
, and lists. In addition, arrays that don’t have a well-defined index set also start at 0. Such cases include array literals or inferred-type arrays formed by capturing a general iterator expression.This change also has a ripple-down effect to features and routines related to these types. For example, varargs arguments are equivalent to tuples in Chapel, so inherit their 0-based indexing. Similarly, queries on rectangular domains and arrays are based on tuples, so their dimensions are now numbered from 0 as well. Certain library routines such as
find()
on strings used to return 0 when no match was found, but now return -1 in order to avoid returning a legal string index.The following sections summarize the rationale for this change and then provide some tips for updating existing Chapel code.
Rationale for 0- vs. 1-based Indexing¶
In the original design of Chapel, we hoped to make the language as neutral to 1- vs. 0-based indexing as possible, to avoid running afoul of the strong emotions that such choices evoke in users when it doesn’t match their preference. As a result, Chapel’s primary types for parallel computation on regular collections of data—namely, its ranges and rectangular domains, as well as rectangular arrays defined by ranges or domains—require users to specify both low and high bounds. Happily, these core features are not affected by this change in Chapel 1.22, so codes relying solely on such features will not require updates.
However, for other types such as tuples and strings, we were forced to make a decision. At the time of Chapel’s inception, the main languages from which we were trying to attract users were C/C++, Java, Fortran, and Matlab. Since half of these languages used 0-based indexing and the other half used 1-based, there didn’t seem to be an obvious best answer. In the end, we decided to go with 1-based indexing on the argument that we were striving to create a productive language, and that counting from 1 is arguably most natural for most people.
Over time, however, the vast majority of newer languages that we look to for users or inspiration—most notably Python, Swift, and Rust—have been almost exclusively 0-based. Meanwhile, very few notable new languages have used 1-based indexing.
Furthermore, when polled, the vast majority of active Chapel users expressed a strong preference for 0-based programming, given the choice (though there were also notable outliers, particularly from the Fortran community). We also realized (a) that Chapel’s design should be more concerned with lowering barriers for existing programmers than for non-programmers; and (b) that even though we had arguably biased the original design in favor of Fortran programmers, most of Chapel’s early adopters have come from C/C++ and Python backgrounds.
Based on this, we undertook an experiment to see what it would take to convert from 1-based to 0-based programming. Reviewing Chapel’s ~10,000 tests and modules resulted in changes to ~1,000 of them. We also updated some significant applications such as Arkouda and Cray HPO. While the overall effort of making the change was not insignificant, it also wasn’t particularly difficult for the most part. Overall, our finding was that in cases where the changes weren’t simply neutral in their impact on style, it almost always benefitted the code in terms of clarity, because there tended to be fewer adjustments of +/- 1 in the code.
For these reasons, we decided to bite the bullet and make the switch now, while we felt we still could, rather than later when it would clearly be too late to do so and cause more of a revolt among our users.
Index-neutral Features¶
This experience also led to a number of new programming features in Chapel 1.21 designed to help write code in more of an index-neutral style. Chief among these are new
.indices
queries on most of the relevant types as well as support for loops over heterogeneous tuples. We also introduced features that we found to be useful in updating code, such as support for open-interval ranges and.first
and.last
queries on enumerated types. To this end, even though Chapel still has cases that require making this 0- vs. 1-based indexing decision, we encourage code to be written in an index-neutral style whenever possible, and believe that most common code patterns can be.Tips for Updating Existing Chapel code¶
The following are some tips for updating codes based on our experiences:
First, updating code is easiest when it has some sort of testing infrastructure that can be used to validate that its behavior is unchanged. If you don’t already have such testing for your code, it may be worthwhile to invest in creating some before attempting this upgrade.
Next, when transitioning code to Chapel 1.22, make sure to compile it with neither
--fast
nor--no-checks
enabled so that bounds checks are turned on in the generated code. In cases where a program is accessing all of the elements of a collection (as is common for tuples) this will help identify data structures that require updates. When you do get an out-of-bounds error, don’t simply update the specific access, but use it as a cue to look through the code for other references to that variable that will also need updating.When possible, try rewriting your updated code to use an index-neutral style of programming. For example, given code like this:
var t: 2*int = ...; var x = t(1), y = t(2); for i in 1..2 do writeln("t(", i, ") = ", t(i));
It would be reasonable to rewrite it like this:
var t: 2*int = ...; var x = t(0), y = t(1); for i in 0..1 do writeln("t(", i, ") = ", t(i));
But arguably preferable to update it like this:
var t: 2*int = ...; var (x, y) = t; for i in t.indices do writeln("t(", i, ") = ", t(i));
If you have a pattern that you’re trying to write in an index-neutral style, but can’t, don’t hesitate to ask for tips.
Some common pitfalls to check for in your code include:
Search for queries on the dimensions of rectangular domains and arrays. For example,
myDomain.dim(1)
,myDomain.low(1)
,myDomain.high(1)
, ormyDomain.stride(1)
will need to be updated to reflect that array dimensions now count from 0 rather than 1. These will result in out-of-bounds errors in cases where you query all dimensions of an array, making them easy to find; but it can be worthwhile to grep your code for such patterns to make sure you don’t miss any.Also search for instances of
find()
orrfind()
that are relying on comparisons to zero/nonzero values, and update them to compare against -1. For example, patterns likeif mystring.find('z')
need to be updated toif mystring.find('z') != -1
.Search for instances of
split()
. A common idiom is to writevar substrs = mystring.split(5);
and then to index into the result usingsubstrs[1]
,substrs[2]
, etc. Since this is an instance of capturing an iterator expression, you’ll either need to subtract one from the indices, or else declare substrs to have a specific type, likevar substrs: [1..5] string = mystring.split(5);
Search for varargs functions and make sure they are updated to use 0-based indexing or index-neutral features.
Search for any calls to
Reflection.getField*()
and update those the cases that use integer indices to reflect 0-based numbering.Look for any calls on lists that use explicit offsets, as these will likely need updates. For example
mylist.pop(1);
will need to becomemylist.pop(0);
Some other common string patterns to look for in your code that may indicate something requiring an update include:
1..
[1]
(1)
[2]
(2)
Think about whether there are other places in your code that compute index values numerically yet which don’t have obvious syntactic cues.
Need Help?¶
If you are able to share your code with us and would like help updating it to Chapel 1.22, please don’t hesitate to ask for help. Given our experience in updating the Chapel code base itself, we have found it fairly easy to update most codes, even when we’re unfamiliar with them.
version 1.21, April 2020¶
Version 1.21 made several improvements related to record initialization, assignment, and deinitialization.
In summary:
Some patterns of default initialization followed by assignment are now converted to initialization. See split initialization.
Some patterns of copy initialization followed by deinitialization are converted to move initialization. See copy elision.
The result of a nested call expression can now be deinitialized at the end of the containing statement. See deinitialization point of nested call expressions.
split initialization¶
Split initialization a new language feature in 1.21 that is described in the language specification - see Split Initialization.
Consider the following example:
var x: myRecord; / default-initialization in 1.20 x = new myRecord(); / assignment in 1.20 -- initialization in 1.21
In 1.21, instead of default-initializing
x
and then assigning to it,x
will be initialized on the second line.Note that split initialization also changes the copy and assignment behavior of
out
intent formal arguments.Occasionally programs that are written to test assignment (separately from copy initialization) need to avoid split initialization. One way to do so is to add a mention of the variable immediately after it is declared, as in the following code:
var x: myRecord; x; / adding this mention prevents split-initialization / instead, x is default-initialized at its declaration point above x = new myRecord();
copy elision¶
Copy elision a new language feature in 1.21. When the last mention of a variable is the source of a copy-initialization, the copy-initialization is replaced by move-initialization.
For example:
class MyClass { var field; proc init(in arg) { this.field = arg; } } proc copyElisionExample() { var a = new myRecord(); var b = a; / now move-initializes `b` from `a` return new MyClass(b); / now move-initializes the field from `b` }
deinitialization point of nested call expressions¶
In 1.20, all variables are deinitialized at the end of the enclosing block. That changed in 1.21. Compiler-introduced temporary variables storing the result of a nested call expression can now be deinitialized at the end of a statement. In particular, results of nested call expressions are now deinitialized at the end of the statement unless the statement is initializing a user variable.
For example:
proc makeRecord() { return new myRecord(); } proc f(arg) { return arg; } proc deinitExample() { f(makeRecord()); / Compiler converts the above statement into / var tmp = makeRecord(); / f(tmp); / In 1.20, tmp is destroyed at the end of the block. / In 1.21, tmp is destroyed at the end of the above statement. var x = f(makeRecord()); / In both 1.20 and 1.21, the temporary storing the result of / `makeRecord()` is deinitialized at the end of the block. }
version 1.20, September 2019¶
Version 1.20 made language changes that address problems with classes.
In summary:
variables of class type can no longer store nil by default but can opt-in to possibly being nil with ?. See nilability changes
certain casts have changed behavior to support nilability changes See nilability and casts
un-decorated class types such as MyClass (as opposed to borrowed MyClass) now have generic management See undecorated classes have generic management
arguments with owned or shared declared type now use const ref default intent rather than in intent. See new default intent for owned and shared
new C
now creates an owned C rather than a borrowed C See new C is owned
nilability changes¶
Previous to 1.20, variables of class type could always store
nil
. In 1.20, only nilable class types can storenil
. Non-nilable class types and nilable class types are different types. A class type expression such asborrowed C
indicates a non-nilable class type.As an aid in migrating code to this change, the flag
--legacy-classes
will disable this new behavior.Consider the following example:
class C { var x:int; } var a: borrowed C = (new owned C()).borrow();
In 1.19, variables of type
borrowed C
could storenil
:var b: borrowed C = nil; var c: borrowed C; a = nil;
The 1.20 compiler will report errors for all 3 of these lines. To resolve the errors, it is necessary to use a nilable class type. Nilable class types are written with
?
at the end of the type. In this example:var a: borrowed C? = (new owned C()).borrow(); var b: borrowed C? = nil; var c: borrowed C?; a = nil;
Implicit conversions are allowed from non-nilable class types to nilable class types.
When converting variables to nilable types to migrate code, there will be situations in which it is known by the developer that a variable cannot be
nil
at a particular point in the code. For example:proc f(arg: borrowed C) { } proc C.method() { } config const choice = true; var a: owned C?; if choice then a = new owned C(1); else a = new owned C(2); f(a); a.method();
Errors on the last two lines can be resolved by writing
f(a!); a!.method();
where here the
!
asserts that the value is notnil
and it can halt if the value isnil
.Note that in
prototype
and implicit file-level modules, the compiler will automatically add!
on method calls with nilable receivers (i.e. in thea.method()
case above).In the above case, a cleaner way to write the conditional would be to create a function that always returns a value or throws if there is a problem. For example:
proc makeC() throws { var a: owned C?; if choice then a = new owned C(1); else a = new owned C(2); return a:owned C; / this cast throws if a stores nil } proc main() throws { var a:owned C = makeC(); f(a); a.method(); }
nilability and casts¶
Because casts to class types should necessarily return something of the requested type, and because many class types now cannot store
nil
, certain patterns involving casts will need to change to work with 1.20.class downcasts¶
In a class downcast, a class is casted to a subtype. If the dynamic type of the variable does not match the requested subtype, the downcast fails. In 1.19, a failed downcast would result in
nil
. In 1.20, a failed downcast will result innil
only if the target type is nilable and will throw an error otherwise.For example:
class Parent { } class Child : Parent { } var p:borrowed Parent = (new owned Parent()).borrow(); var c:borrowed Parent = (new owned Child()).borrow(); writeln(c:Child?); / downcast succeeds writeln(c:Child); / downcast succeeds writeln(p:Child?); / this downcast fails and results in `nil` writeln(p:Child); / this downcast fails and will throw a ClassCastError
casting C pointers to classes¶
Casts from
c_void_ptr
to class types were previously allowed. However, sincec_void_ptr
can storeNULL
, this case needs adjustment following the nilability changes. Additionally, sincec_void_ptr
refers to a C pointer, and C pointers are manually managed (i.e. you callfree
on them at the appropriate time), it makes the most sense for casts fromc_void_ptr
to end up with an unmanaged type.Consider the following example:
class C { var x:int; } var myC = new owned C(); var ptr:c_void_ptr = myC.borrow(); / store the instance in a C ptr
Now we can cast from
ptr
to the class type:var c = ptr:C; / cast from a C pointer to the borrowed type
This example would work in 1.19. In 1.20, it needs to be updated to cast to
unmanaged C?
:var c = ptr:unmanaged C?;
As with other values of type
unmanaged C?
, from there it can:be borrowed, e.g.
c.borrow()
have
!
applied to convert to a non-nilable value or halt, e.g.c!
be cast to a non-nilable type, throwing if it is
nil
, e.g.c:borrowed C
undecorated classes have generic management¶
Undecorated classes now have generic management. As an aid in migrating code to this change, the flag
--legacy-classes
will disable this new behavior.Supposing that we have a
class C
declaration as in the following:class C { var x:int; }
Code using
C
might refer to the typeC
on its own or it might use a decorator to specify memory management strategy, as inborrowed C
.The type expression
C
was the same asborrowed C
in 1.18 and 1.19 but now means generic management. For example, in the following code:var myC:C = new owned C();
myC
previously had typeborrowed C
, and was initialized using including an implicit conversion fromowned C
toborrowed C
. In 1.20,myC
has typeowned C
. Since the variable’s type expression is generic management, it takes its management from the initializing expression.This change combines with the nilability changes described above to prevent compilation of existing code like the following:
var x:C;
Knowing that
C
now cannot storenil
, one might try to update this program to:var x:C?;
However this does not work either.
C?
indicates a nilable class type with generic management, and a variable with generic type cannot be default-initialized.To update such a variable declaration to 1.20, it is necessary to include a memory management decorator as well as
?
. For example:var x:borrowed C?;
The resulting variable will initially store
nil
.new C is owned¶
Supposing that C is a class type, new C() was equivalent to new borrowed C() before this release - meaning that it resulted in something of type borrowed C. However, it is now equivalent to new owned C() which produces something of type owned C.
version 1.18, September 2018¶
Version 1.18 includes many language changes that address problems with classes.
In summary:
constructors are deprecated and replaced with initializers See initializers replace constructors
memory management for class types has changed See class memory management
override is now required on overriding methods See overriding methods must be marked
initializers replace constructors¶
Code that contained user-defined constructors will need to be updated to use an initializer. For example:
record Point { var x, y: real; proc Point() { x = 0; y = 0; writeln("In Point()"); } proc Point(x: real, y: real) { this.x = x; this.y = y; writeln("In Point(x,y)"); } } var a:Point; var b = new Point(1.0, 2.0);
will now compile with deprecation warnings. Here is the same program updated to use initializers:
record Point { var x, y: real; proc init() { x = 0; y = 0; writeln("In Point.init()"); } proc init(x: real, y: real) { this.x = x; this.y = y; writeln("In Point.init(x,y)"); } } var a:Point; var b = new Point(1.0, 2.0);
The change to initializers is much more than a change in the name of the method. See the language specification for further details.
class memory management¶
Before 1.18, if
C
is a class type, a variable of typeC
needed to be deleted in order to prevent a memory leak. For example:class C { var x: int; } proc main() { var instance: C = new C(1); delete instance; }
Version 1.18 introduced four memory management strategies that form part of a class type and are used with new expressions:
owned C
owned
classes will be deleted automatically when theowned
variable goes out of scope, but only oneowned
variable can refer to the instance at a time. Such instances can be created withnew owned C()
.shared C
shared
classes will be deleted when all of theshared
variables referring to the instance go out of scope. Such instances can be created withnew shared C()
.borrowed C
refers to a class instance that has a lifetime managed by another variable. Values of type
borrowed C
can be created withnew borrowed C()
, by coercion from the other classC
types, or by explicitly calling the.borrow()
method on one of the other classC
types.new borrowed C()
creates a temporary instance that will automatically be deleted at the end of the current block.unmanaged C
the instance must have delete called on it explicitly to reclaim its memory. Such instances can be created with
new unmanaged C()
.
Further note that the default is
borrowed
, that is:C
is now the same as
borrowed C
new C()
is now the same as
new borrowed C()
Now, back to the example above. There are several ways to translate this program.
First, the most semantically similar option is to replace uses of
C
withunmanaged C
:class C { var x: int; } proc main() { var instance: unmanaged C = new unmanaged C(1); delete instance; }
Using
unmanaged
allows a Chapel programmer to opt in to manually managing the memory of the instances.A reasonable alternative would be to translate the program to use
owned C
:class C { var x: int; } proc main() { var instance: owned C = new owned C(1); / instance will now be automatically deleted at the end of this block }
If the program does not explicitly use
owned C
, it can rely onnew C()
being equivalent tonew borrowed C()
:class C { var x: int; } proc main() { var instance: C = new C(1); / instance will now be automatically deleted at the end of this block }
See the Class New section in the Classes chapter of the language specification for more details.
overriding methods must be marked¶
Before 1.18, a class inheriting from another class can create an overriding method that is a candidate for virtual dispatch:
class Person { var name: string; proc greet() { writeln("Hello ", name, "!"); } } class Student: Person { var grade: int; proc greet() { writeln("Hello ", name, ", welcome to grade ", grade); } } proc main() { var person: Person = new Student("Jeannie", 5); person.greet(); / uses the run-time type of person (Student) / and virtually dispatches to Student.greet() }
Now such overriding methods must be marked with the override keyword:
class Person { var name: string; proc greet() { writeln("Hello ", name, "!"); } } class Student: Person { var grade: int; override proc greet() { writeln("Hello ", name, ", welcome to grade ", grade); } } proc main() { var person: Person = new Student("Jeannie", 5); person.greet(); / uses the run-time type of person (Student) / and virtually dispatches to Student.greet() }
version 1.15, April 2017¶
Version 1.15 includes several language changes to improve array semantics.
In summary:
arrays are always destroyed when they go out of scope and in particular will not be preserved by use in begin. See array lexical scoping.
the array alias operator => has been deprecated in favor of creating references to an array or a slice of an array with ref or const ref. See array alias operator deprecated.
arrays now return by value by default instead of by ref. See arrays return by value by default.
arrays now pass by ref or const ref by default, depending on whether or not the formal argument is modified. See array default intent.
Additionally, the default intent for record method receivers has changed:
the method receiver for records is passed by ref or const ref by default, depending on whether or not the formal argument is modified. See record this default intent.
array lexical scoping¶
As described in the language changes for 1.12 in lexical scoping, using arrays beyond their scope is a user error. While such a program was in error starting with Chapel 1.12, such a pattern worked until Chapel 1.15.
For example, this program will probably crash in Chapel 1.15:
proc badBegin() { var A: [1..10000] int; begin { A += 1; } / Error: A destroyed here at function end, but the begin could still / be using it! }
Similarly, using a slice after an array has been destroyed is an error:
proc badBeginSlice() { var A: [1..10000] int; / slice1 sets up a slice using the => operator / note that the => operator is deprecated (see below) var slice1 => A[1..1000]; / slice2 sets up a slice by creating a reference to it ref slice2 = A[1..1000]; / either way, using the slice in a begin that can continue / after the function declaring the array exits is an error begin { slice1 += 1; slice2 += 1; } / Error: A destroyed here at function end, but the begin tries to / use it through the slices! }
array alias operator deprecated¶
The array alias operator, =>, has been deprecated in Chapel 1.15. Previously, the supported way to declare one array that aliases another (or a slice of another) was to use =>. Now, the supported way to do that is to use a ref or const ref variable:
For example, before Chapel 1.15 you might have written:
/ pre-1.15 var A:[1..10] int; / set up a const alias of A const alias => A; / set up a mutable slice of A var slice => A[2..5]; / set up a re-indexing slice of A var reindex:[0..9] => A;
In Chapel 1.15, use ref or const ref to create the same pattern:
var A:[1..10] int; / set up a const alias of A const ref alias = A; / set up a mutable slice of A ref slice = A[2..5]; / set up a re-indexing slice of A ref reindex = A.reindex({0..9});
arrays return by value by default¶
Before Chapel 1.15, returning an array would return the array by reference. Now arrays return by value by default. That is, the act of returning an array can make a copy:
var A: [1..4] int; proc returnsArray() { return A; } ref B = returnsArray(); B = 1; writeln(A); / outputs 1 1 1 1 historically / outputs 0 0 0 0 after Chapel 1.15
This behavior applies to array slices as well.
The old behavior is available with the ref return intent. Note though that returning a ref to a local array is an error just like it is an error to return a local int variable by ref.
proc returnsArrayReference() ref { return A; }
array default intent¶
Before 1.15, the default intent for arrays was ref. The rationale for this feature was that it was a convenience for programmers who are used to modifying array formal arguments in their functions. Unfortunately, it interacted poorly with return intent overloading. Additionally, the implementation had several bugs in this area.
The following example shows how it might be surprising that return intent overloading behaves very differently for arrays than for other types. As the example shows, this issue affects program behavior and not just const-checking error messages from the compiler.
/ First, let's try some of these things with an / associative array of ints: { var D:domain(int); var A:[D] int; / This adds index 1 to the domain, implicitly A[1] = 10; writeln(D.member(1)); / outputs `true` / This will halt, because index 2 is not in the domain /var tmp = A[2]; / This will also halt, for the same reason /writeln(A[3]); } / Now, let's try the same things with an array of arrays: { var D:domain(int); var AA:[D] [1..4] int; var value:[1..4] int = [10,20,30,40]; / This adds index 4 to the domain, implicitly AA[4] = value; writeln(D.member(4)); / outputs `true` / This will halt, because index 5 is not in the domain /var tmp = AA[5]; / It seems that this *should* halt, but it does not (pre 1.15) / Instead, it adds index 6 to the domain writeln(AA[6]); writeln(D.member(6)); / outputs `true` ! }
See GitHub issue #5217 for more examples and discussion.
In order to make such programs less surprising, version 1.15 changes the default intent for arrays to ref if the formal argument is modified in the function and const ref if not. As a result, the above example behaves similarly for an associative array of integers and an associative array of dense arrays.
For example, in the following program, the default intent for the formal argument x is ref:
proc setElementOne(x) { / x is modified, so x has ref intent x[1] = 1; } var A:[1..10] int; setElementOne(A);
In contrast, in the following program, the default intent for the formal argument y is const ref:
proc getElementOne(y) { / y is not modified, so y has const ref intent var tmp = y[1]; } const B:[1..10] int; getElementOne(B);
record this default intent¶
Before 1.15, the default intent for the implicit this argument for record methods was implemented as ref but specified as const ref. In 1.15, this changed to ref if the formal this argument is modified in the body of the function and const ref if not.
See GitHub issue #5266 for more details and discussion.
record R { var field: int; proc setFieldToOne() { / this is modified, so this-intent is ref this.field = 1; } proc printField() { / this is not modified, so this-intent is const ref writeln(this.field); } }
version 1.13, April 2016¶
ref return intent¶
Previous versions of Chapel included an implicit setter param of type bool for ref return intent functions. In addition, the compiler created a getter and setter version of each ref return intent function. The getter version would return an rvalue, and the setter version would return an lvalue by ref. For example:
var x = 1; proc refToX() ref { if setter then return x; / setter version else return 0; / getter version } refToX() = 3; / uses the setter version writeln(x); / prints 3 var tmp = refToX(); / uses the getter version writeln(tmp); / prints 0
This functionality has changed with version 1.13. It is still possible to write a getter and a setter, but these must be written as pair of related functions:
var x = 1; / setter version proc refToX() ref { return x; } / getter version proc refToX() { return 0; } refToX() = 3; / uses the setter version writeln(x); / prints 3 var tmp = refToX(); / uses the getter version writeln(tmp); / prints 0
In some cases, when migrating code over to the new functionality, it is useful to put the old ref return intent function into a helper function with an explicit param setter argument, and then to call that function from the getter or setter.
version 1.12, October 2015¶
lexical scoping¶
Prior to version 1.12 of Chapel, variables could be kept alive past their lexical scopes. For example:
{ var A: [1..n] real; var count$: sync int; var x: real; begin with (ref x) { ... A ...; ... count$ ...; ... x ...; } / ^^^ this task and its references to A, count$, and x could outlive / the scope in which those variables are declared. } / So, previously, Chapel kept these variables alive past their / logical scope.
Disadvantages of this approach included:
It moves logical stack variables (like x and count$ above) to the heap.
It complicates memory management by incurring reference counting overhead—or causing memory leaks in cases where reference counting hadn’t been added.
It was not particularly valued or utilized by users.
It was arguably surprising (“x still exists even though it left scope?”).
As of Chapel 1.12 (and moreso in subsequent releases), the implementation no longer provides this property. Instead, it is a user error to refer to a variable after it has left scope. For example:
var flag$: sync bool; / flag$ starts empty { var x: real; begin with(ref x) { / create task referring to x flag$; / block task until flag$ is full ... x ... / user error: access to x occurs after it leaves scope } / end task } / x`s scope ends flag$ = true; / fill flag$ only after x's scope closes
Code that refers to lexically scoped variables within tasks in this manner should use sync variables or blocks in order to guarantee the tasks’s completion before the enclosing block exits. Note that the more commonly used cobegin, coforall, and forall statements already guarantee that the tasks they create will complete before the enclosing block exits.
version 1.11, April 2015¶
forall intents¶
In previous versions of Chapel, the bodies of forall-loops have referred to all lexically visible variables by reference. In this release of Chapel, such variables are treated more consistently with the task intent semantics and syntax introduced in versions 1.8 and 1.10 respectively (described below).
Specifically, prior to this release, a loop like the following would represent a data race:
var sum = 0.0; forall a in A do sum += a;
since multiple iterations of the loop could execute simultaneously, read the identical value from the shared variable
sum
, update it, and write the result back in a way that could overwrite other simultaneous updates.Under the new forall intent semantics, such variables are treated as though they are passed by “blank intent” to the loop body (so
const
for variables of scalar type likesum
, preventing races in such cases). This mirrors the task intent semantics for variables referenced within begin,cobegin
, andcoforall
constructs. As in those cases, a user can specify semantics other than the default via a with-clause. For example, to restore the previous race-y semantics, one could write:var sum = 0.0; forall a in A with (ref sum) do sum += a;
(Of course, the safe way to write such an idiom would be to use a reduction, or a synchronization type like
sync
oratomic
).type select statement¶
Chapel has traditionally supported a
type select
statement that was like aselect
statement for types. However, this seemed inconsistent with the fact that other constructs likeif...then
operate on types directly. For that reason, this release removed support fortype select x
. Instead, use the equivalentselect x.type
.version 1.10, October 2014¶
task intents syntax¶
Task intent clauses were added to Chapel in version 1.8 to support passing variables by reference into tasks. Since then, the need to pass variables by other intents and into other parallel constructs has arisen. But, the previous syntax was problematic to extend to other intents, while also generating syntactic ambiguities for other additions we wished to make to the language.
For these reasons, a new task intent syntax was designed to cleanly support intents other than
ref
(particularly in looping contexts), to address the pending ambiguity, and to better reflect the similarity of task intents to formal argument lists. Where previously, task constructs could be followed by aref
clause, they can now be followed by awith
clause that takes a list of intents and variables, specifying how to pass them into the task.Thus, where one would have previously written:
begin ref(x) update(x); cobegin ref(x, y) { process(x); process(y); } coforall x in A ref(y) { process(x, y); }
you would now write:
begin with (ref x) update(x); cobegin with(ref x, ref y) { process(x); process(y); } coforall x in A with (ref y) { process(x, y); }
As of the 1.10 release, only
ref
intents are supported, though we plan to expand this set of intents for the 1.11 release while also extending forall-loops to support task intents.‘var’ function return intents changed to ‘ref’¶
A
var
function return intent has traditionally been used to indicate that a call to the function (referred to as a var function) could appear in either an r-value or l-value context. Thevar
keyword was chosen since the function could be used in the same contexts as a variable could.Since that time, the
ref
keyword has been introduced into Chapel to support passing variables by reference to functions. Since returning an expression by reference supports similar capabilities asvar
functions require, while also being less unusual/more orthogonal, this release replacesvar
function return intents withref
intents.Thus, where one would previously write:
proc getFoo() var { ... }
now you would write:
proc getFoo() ref { ... }
The
var
as a return intent is deprecated and generates a warning for the current release, after which it will be removed.version 1.9, April 2014¶
operator precedence changes to benefit common cases¶
Historically, Chapel’s operator precedence choices have tended to follow the lead of C for operators that are common to both languages, figuring that following an established convention would be better than forging our own path.
With this change, we modified the precedence of bitwise operators to better reflect what we think it intuitive to users and correct what is viewed in many circles to be a regrettable mistake in C. At the same time, we changed the binding of
in
and..
to support some other Chapel idioms more naturally, like1..10 == 1..10
. To see the current operator precedence, refer to the Quick Reference sheet.improved interpretation of {D}¶
Historically, for a domain D, Chapel has interpreted
{D}
as being equivalent toD
, inheriting a precedent of sorts set by the ZPL language, and dating from a time when we used square brackets for both domain literals and array types.With this change, we began interpreting
{D}
as a domain literal with a single index,D
(i.e., an associative domain of domains). Associative domains of domains are not yet implemented in the language, so the new syntax is not yet useful, but at least the incongruity of ignoring the curly brackets has been removed.version 1.8, October 2013¶
task functions and intents; ref-clauses Chapel has three constructs for creating tasks:
begin
,cobegin
, andcoforall
. Historically, variable references within tasks followed standard lexical scoping rules. For example, the following code:var x = 0; begin writeln(x); x += 1;
could print either the value 0 or 1, depending on whether the
writeln()
task was executed before or after the increment ofx
.With this change, we view the creation of a task as an invocation of a task function — a compiler-created function that implements the task. Any references to variables outside of the task’s scope (like
x
in the example above) are treated as implicit arguments to the task function, passed by blank intent.Thus, when
x
is an integer, as in the above code, the task will always print the value of 0, even if the increment ofx
is executed before thewriteln()
task, since the value ofx
will have been passed to the task function by blank intent (implying aconst
copy for integer arguments). In contrast, if x were a sync variable in the example above, the blank intent would cause it to be passed by reference to the task, permitting the task to see either of the values 0 or 1.To return to the previous behavior, a ref-clause can be added to the tasking construct to indicate that a variable should be passed to the task function by reference rather than blank intent. For example, the following code:
var x = 0; begin ref(x) writeln(x); x += 1;
would revert to the previous behavior, even if
x
were an integer.For more information on this feature, please refer to the Task Intents section of the Task Parallelism and Synchronization chapter of the language specification.
version 1.6, October 2012¶
domain literals¶
Chapel’s domain literals were historically specified using square brackets, based on ZPL’s region syntax. Thus
[1..m, 1..n]
represented an m × n index set.In this change, we made domain literals use curly brackets in order to reflect their role as sets of indices, and also to make square brackets available for supporting array literals. Thus,
{1..m, 1..n}
is an m × n index set,[1.2, 3.4, 5.6]
is a 3-element array of reals and[1..m, 1..n]
is a 2-element array of ranges.Emacs users working on updating existing code can use the following recipe to update old-style domain literals to the new syntax:
M-x query-replace-regexp: \([=|,] *\)\[\(.*?\)\]\([;|)]\) with: \1{\2}\3
zippered iteration¶
Zippered iteration in Chapel was traditionally supported simply by iterating over a tuple of values. For example, forall
(i,a)
in(1..n, A)
would iterate over the range1..n
and the n-element arrayA
in a zippered manner.In this change, we introduced the zip keyword to make these zippered iterations more explicit and to permit iteration over a tuple’s values directly. Thus, the zippered iteration above would now be written:
forall (i,a) in zip(1..n, A)
ignoring tuple components/underscore¶
Overtime, the mechanism used to ignore a tuple component when destructuring a tuple has changed. Originally, an underscore was used to drop a value on the floor. For example, given a 3-tuple
t
, the first and last components could be stored inx
andz
, dropping the second component on the floor using:var (x, _, z) = t;
. In version 1.1 (Apr 2010), we changed this to use a blank space instead of an underscore, for simplicity and to permit underscore to be used as an identifier name. Thus, the example above would have been written asvar (x, , z) = t;
during this time period.However, in 2012, we changed back to using the underscore again in order to support the specification of 1-tuples using a dangling comma, similar to Python. Thus, dropping a tuple component is expressed as
var (x, _, z) = t;
again while(1.2, )
is a 1-tuple of reals.version 1.4, October 2011¶
function declaration keywords¶
Prior to this change, the keyword
def
was used to define both procedures and iterators; the compiler inspected the body of the function for yield statements to determine whether it was a procedure or an iterator.In this change, we introduced the
proc
anditer
keywords to distinguish between these two cases for the sake of clarity, to avoid mistakes, to support semantics that clearly distinguish between these cases, and to better support specifying interfaces.